Controlling Chaos in Chaotic Neural Networks

Shin Mizutani

NTT Human Interface Laboratories, Yokosuka, Japan 238-03

Takuya Sano

NTT Optical Network Systems Laboratories, Yokosuka, Japan 238-03

Tadasu Uchiyama and Noboru Sonehara

NTT Human Interface Laboratories, Yokosuka, Japan 238-03

SUMMARY

This work demonstrates the control of chaos in cha-

otic neural networks. Chaotic neural networks, which were

proposed by Aihara and others, consist of chaotic neuron

models, and are based on research on the giant axon of

squids and study of the Hodgkin-Huxley equation. They

show a chaotic response that cannot be expressed by con-

ventional neuron models. Although research has been per-

formed on utilizing this chaotic response for active search

of associative memory, it is difficult to determine when to

stop the chaotic dynamics. Therefore, we have recently

investigated chaotic control methods that had been the

subject of previous research. Among these control methods,

there is a method in which unstable period points are

stabilized by multiplying the exponent or exponential func-

tion by the system parameters. We used an improved ver-

sion of this method for high-dimension system control. For

simplicity, we describe the control of networks that are

coupled with the first order nearest neighbors and have

global homogeneous coupling. We confirm that by using

this control method, the unstable period points in the net-

work can be stabilized and control of chaos is possible. With

this method, stabilization is also possible when noise is

added to the system. ©1998 Scripta Technica, Electron

Comm Jpn Pt 3, 81(8): 73�82, 1998

Key words: Chaotic neural network; high-dimen-

sional system; chaotic control; stabilization; exponential

function feedback.

1. Introduction

In recent years, research on neuro-computing has

been intensified in order to achieve flexible intelligent

information processing networks like those of the brain.

The dynamics of equilibrium due to the energy function

expression have been modeled by using associative mem-

ory. In this work, the spin-glass analogy, an extremely

simple model of the neuron, has been applied to real neuron

cells. However, because this model is an oversimplification,

the actual characteristics of the neuron cell cannot be rep-

resented in this neuron model. The chaotic neuron model

proposed by Aihara and others is based on research on the

giant axon of the squid and on the study of the Hodgkin-

Huxley equation. Using this model, chaotic responses not

represented by the conventional neuron model were dem-

CCC1042-0967/98/080073-10

© 1998 Scripta Technica

Electronics and Communications in Japan, Part 3, Vol. 81, No. 8, 1998Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J79-A, No. 9, September 1996, pp.1562�1570

73

onstrated [1]. A chaotic neural network constructed using

the chaos neuron and with an autocorrelation matrix of the

weighted coupling of memory patterns can aperiodically

recall stored patterns [2]. Associative memory using these

dynamics can achieve the ability to dynamically search

memory. However, a network that develops temporally

from a certain initial condition may change its state even if

the state comes close to the memory pattern. It is difficult

to judge when to stop the chaotic dynamics and when the

use of associative memory is desired. This is a problem that

is always faced when utilizing chaos in memory searches.

Several methods have been proposed in order to avoid this

problem. In one method, the dynamics of the network are

changed to exit chaos states when the network state comes

close to one of the memory patterns [3].

In this paper we examine chaos control, developed in

recent research, as a method for avoiding the aforemen-

tioned problem. In these chaos control methods, in view of

the fact that the chaos attractor typically has embedded

within it an infinite number of unstable periodic orbits, the

system state is coerced into the target unstable periodic orbit

by perturbing the parameters of the system equation or by

changing the equation itself. In the method in which pa-

rameter perturbations are used, the state is coerced into the

path to an unstable periodic orbit by bringing the stable

periodic orbit onto a stable manifold. In the method using

equation change, control is made possible by changing the

unstable periodic orbit to a stable periodic orbit.

In this work, the control method of using changes in

equations to coerce a network with an unstable orbit into a

stable periodic point is used. For simplicity we demonstrate

chaos control in a chaotic neural network with the simplest

structure. This paper has the following form. In section 2

the chaotic neuron model and chaotic neural network are

described. Section 3 gives a brief overview of various chaos

control methods that have been proposed. In section 4 a

method based on the exponential feedback control (EFC)

method for controlling the high-dimensional chaos sys-

tems, such as neural nets is presented. In section 5 control

by a chaotic neuron based on the proposed method is

shown. In section 6, similarly, the formulation of chaos control

for chaotic neural networks is described and numerical results

are presented. In section 7, future areas of research on the

control of complex neural networks are discussed.

2. Chaotic Neuron Model and the Chaotic

Neural Network

2.1. Chaotic neuron model

Development of the chaotic neuron model was based

on research on the squid giant axon and study of the

Hodgkin-Huxley equation [1]. In this model, these phe-

nomena are described qualitatively, and expressed in a

discrete time structure. The simplest case of the model is

shown in the following equations:

Here, y(t) is the internal state of the neuron at time t, x(t) isthe output of the neuron at time t, k is the dumping constant,

a is the refractory term (a ³ 0), a is a parameter based on

the input and the threshold, and e is the gradient of the

sigmoid function.

By changing the parameter a, the internal condition

of the chaotic neuron y(t) and the output x(t) become

chaotic. The bifurcation diagram and Lyapunov exponent

of the internal state y(t) when the parameter a is changed

are shown in Fig. 1. The other parameters are: k = 0.7, a =

1.0, and e = 0.02. The horizontal axis of Fig. 1 is the

parameter a, the vertical axis of (a) is the neuron internal

state y(t), and the vertical axis of (b) is the Lyapunov

exponent. The Lyapunov exponent is given by the following

equation:

In numerical simulations, the initial 2000 time steps are

excluded. The next 3000 steps are used to determine the

Lyapunov exponent.

For the following calculations, the chaotic neuron

parameters used are k = 0.7, a = 1.0, a = 0.35, and e = 0.02.

This parameter set produces chaos with the Lyapunov ex-

ponent l = 0.38.

2.2. Chaotic neural network

The simplest network in which the chaotic neurons

are coupled can be described by the following equations:

In this research, for simplicity, a neural network in which

the chaotic neurons have homogeneous coupling w is inves-

tigated. This is represented by the following equation:

(1)

(2)

(3)

74

Here, S is calculated for the physically closest neuron,

excluding the neuron itself, in the nearest neighbor coupling

case Nc = (the nearest neuron number + the neuron itself),

and for all neurons, excluding the neuron itself, in the global

coupling case, Nc = N (all neurons). For the nearest neigh-

bor coupling, periodic boundary conditions are used.

3. Methods for Controlling Chaos

Chaos control was developed in the course of physi-

cal research on non-linear systems and in recent years has

begun to be applied to control of laser oscillation, chemical

reactions, irregular pulses, and so on. It has also been

attracting interest in engineering fields. Control methods

can be subdivided into those that use feedback and those

that do not. An example of a method that uses feedback is

the OGY method proposed by Ott, Grbogi, and Yorke [4].

Form this method, the occasional proportional feedback

(OPF) method [5, 6], the continuous feedback method [7],

the EFC method [8], and other methods have been proposed

that are easier to apply than the OGY method. In these chaos

control methods, taking into account the fact that the chaos

attractor typically has embedded within it an infinite num-

ber of unstable periodic orbits, the system state is coerced

into the target unstable periodic orbit by feedback.

The OGY method is based on the Poincaré map. In

this method, we add a perturbation to the equation parame-

ters when the state of the system approaches the target

unstable periodic point; thus we bring the state onto the

stable manifold that is approaching the unstable periodic

point. In this manner, the system state is coerced onto an

unstable periodic point. The added perturbation is obtained

by linearly approximating the neighbor of the target unsta-

ble periodic point, and is extremely small [4].

The OPF method can be applied when the return map

obtained from the system becomes simple in the neighbor-

hood of the target unstable point. It is a modified OGY

method based on the return map. Similarly to the OGY

method, it controls by performing perturbation based on

feedback proportional to the time-series for the system

equation. This control is performed when the time-series

takes values in a certain region and the size of the feedback

is calculated from the return map [5, 6].

In these methods, the system state is coerced onto the

unstable periodic point by performing feedback only when

the system state is near the target unstable periodic point so

that a linear approximation model can be used. After the

system state reaches the unstable periodic point, control is

performed using perturbations only when the state is off the

point.

In the chaos control system using continuous feed-

back proposed by Pyragas, linear feedback continues to be

added to the original system equation during control. By

changing the equation itself, the target unstable periodic

point is changed to a stable periodic point. In this method,

control feedback can be added even if the system path is not

near the target. [7].

The EFC method, proposed by Gadre and Varma, is

a similar method. In this method, control is performed by

multiplying the feedback by the system parameter [8]. The

multiplication of the parameter by the feedback in this

method is equivalent to adding control feedback if a Taylor

expansion is used. Therefore, this nonlinear feedback is

different from the continuous feedback method.

Fig. 1. Bifurcation diagram and Lyapunov exponent of

a chaotic neuron for parameter a.

(4)

75

Chaos control of chaotic neurons may be possible by

any of these methods. However, for networks with high

dimension, it is rare for the system state to approach the

target unstable periodic point, so the time interval before

feedback can be performed using the OGY and OPF meth-

ods is extremely long. In contrast, for the continuous feed-

back method and the EFC method, feedback can be added

even when the system state is far from the target point. Time

is not wasted during the interval in which the state ap-

proaches the target, so these methods are better for reducing

the time required for control. For these two methods, a

feedback limit is set such that the system does not diverge.

When these methods are applied to chaotic neuron map-

ping, the EFC method, which uses exponential feedback,

has a larger basin, so we used this method. Furthermore, to

apply the EFC method to the control of high-dimension

systems, several changes were made. For the applicable

systems, the improved EFC method for the case of mapping

is described in the following section.

4. The Improved EFC Method for Control

of High-dimension Chaos

First, an N dimensional discrete dynamic system is

expressed by the difference equation shown below:

Here x(t) º (x1(t), x2(t), . . . , xN(t)) are variables, and

m º (m1, m2, . . . , mM) are parameters.

For example, parameter mj is selected, and the vari-

able xi(t) related to this parameter is used to obtain the

following exponential function for controlling the period-1

point x_®(t1) = (x

_1(t1), x

_2(t1), . . . , x

_N(t1)).

Here, Ki is a parameter that expresses the control strength.

This feedback can become too large, depending on the

initial conditions, which may result in a diverging system,

thus a limit is set on the amount of feedback. System control

without divergence is then possible by using this limit.

The dynamics of the system modulated with this term

are

In this method, if the system state x(t) reaches x_®(t1), the

variable xi(t) becomes equivalent to xi__(t1) and the amount

of feedback becomes unity, so that the effect of feedback

disappears. If the system state moves away from the peri-

odic point due to fluctuation, then the exponential function

term becomes effective again. The feedback variables and

the combination of parameters must be chosen with refer-

ence to the system target.

For control of an unstable point with period L, the

feedback is periodically varied in accordance with target

x_i(t1). At time t, to control the target x

_i(t1), the term

is multiplied by the parameter mj.

Control is performed using the following procedure.

1. Obtain the unstable periodic point to be stabi-

lized (the reference point) (x_® (t1), x

_® (t2), . . . , x

_® (tL).

2. Obtain the Jacobian dG®

control / dx®(t) eigenvalue of

the system equation to which feedback for stabilization is

added.

3. Determine the control parameter Ki(tl) so that the

absolute values of all the eigenvalues are smaller than unity

in order to stabilize the unstable periodic point.

4. Set a limit to the amount of feedback that is

suitable to the applicable system to prevent system diver-

gence due to feedback.

For this method, the setting for the feedback limit and

the size of the basin have not been discussed. Therefore, for

all dynamic systems, the system is not controllable for

every initial condition and limits.

5. Chaos Control for the Chaotic Neuron

The equation of the controlled chaotic neuron is

where y(t) and K(t) are the reference points at time t and the

control parameter, respectively. FL and FH are the feedback

lower and upper limits, respectively. Control of a chaotic

neuron using this method is shown in Fig. 2. This is the

control for a period-1 point y(t) = �0.012063. (a) is the time

change of y(t), and (b) is the time-dependence of the feed-

back F(t). The parameters are K(t) = 32.0, y(t) = �0.012063,

and FH = 1.2. The lower limit FL is not set. Control begins

(5)

(6)

(7)

(8)

(9)

76

at t = 200, but F(t) soon becomes unity, and control to a

period-1 point can be performed.

Mapping of the controlled chaotic neuron is shown

in Fig. 3b. When the initial value of the variable y(t) is

changed in increments of 0.001 on the interval [�10.0,

10.0], the stable periodic point is always the period-1 point.

It follows that the unstable periodic points of the original

mapping can be changed into stable periodic points by

feedback.

6. Chaos Control for a Chaotic Neural

Network

6.1. Formulation

The controlled chaotic neuron network can be ex-

pressed as follows:

Here, S is the sum of the nearest neurons in the case of

nearest neighbor coupling or the sum of all neurons except

itself in the case of global coupling.

It is theoretically applicable to the network with

heteogeneous coupling, following section 4.

Fig. 2. Control of a chaotic neuron.

(10)

Fig. 3. Chaotic neuron map.

77

6.2. Numerical results

Figures 4, 5, and 6 show numerical results obtained

by using this method. The network dimension is N = 100.

The initial value of yi(t) is a random number on [0, 1], w =

0.02 for the case of nearest neighbor coupling, and w =

0.001 in the case of global coupling. Also, ai = a = 0.35.

The maximum Lyapunov exponent lmax is 0.49 for nearest

neighbor coupling and 0.52 for global coupling, which is

chaotic. Control commenced at time t = 200. The reference

point and the control parameters are as follows, for all

numerical calculations.

Period-1 point: The reference point is period-1

point yi(t) = �0.012063 (xi(t) = 0.353619). The control

parameter is Ki(t) = 32.0. The limit of the feedback is FH =

1.2 for the upper limit. The lower limit FL is not set.

Period-2 points: The reference points are period-2

points yi(t) = 0.031316, yi(t + 1) = �0.455263, (xi(t) =

0.827184, xi(t + 1) = 0.000000). The control parameter is

Ki(t) = 20.0, Ki(t + 1) = �1.0. The upper limit on feedback

is FH = 1.2. The lower limit FL is not set.

Figure 4 shows the result for 1-dimensional nearest

neighbor coupling with unit time period. Figure 4(a) shows

a point plot with the x-axis being the time t and the y-axis

being the output of neuron xi(t). The size of the dots indicate

the size of xi(t). In Fig. 4(b), all neuron outputs xi(t) are

shown as broken lines with the x-axis being the time t and

the y-axis being the output xi(t). At some time after t = 200,

all neurons can be coerced into a period-1 point. Figure 5

shows the case for 1-dimensional nearest neighbor coupling

and time period 2 in the same manner as Fig. 4. In this case,

noise interval [�0.01, 0.01] is added to the right hand side

of the expression for y(t), Eq. (10). In this case also, it can

be coerced into period-2 points.

Figure 6 shows the case with the global coupling; (a)

shows control to a period-1 point, and (b) control to period-

2 points. The graphs can be read in the same manner as

previously. However, the order of the neurons on the y-axis

is meaningless in global coupling. In this case also, it can

be coerced to the period points.

To investigate the Ki(t) (= K) dependence of this con-

trol method, we examined the case of nearest neighbor

coupling for time period 1 and calculated the absolute value

of the Jacobian eigenvalue of the controlled system and the

maximum Lyapunov exponent. The absolute value of Jaco-

bian eigenvalue of the system is smaller than unity when

Fig. 4. Controlling chaos to stabilize a period-1 point of a neural network with 1-dimensional nearest neighbor coupling.

78

30.4084 < K < 33.5103. In this region, the points of time

period 1 are locally stable. However, when K is changed in

increments of 0.05 in the region [30.5, 33.5] and the con-

trolled system develops temporally from some initial value,

the maximum Lyapunov exponent is as shown in Fig. 7(a).

It follows that the region of K is controllable except for a

few points. This region of controllable K and the existence

of a few uncontrollable points did not change greatly when

the initial value was varied. To investigate the dependence

of the feedback limits, FL and FH, we considered the case

of nearest neighbor coupling in time period 1, and calcu-

lated the maximum Lyapunov exponent of the controlled

system. The lower limit FL can be omitted because the

feedback is in the form of an exponential function. There-

fore, we considered the upper limit FH that prevents diver-

gence of the controlled system. Figure 7(b) shows the

maximum Lyapunov exponent when FH is varied in incre-

ments of 0.05 in the interval [1.0, 2.0]. It is controllable in

the interval [1.15, 1.7]. However, in the same manner as

seen for K-dependence, a few uncontrollable points exist

depending on the initial value. It follows that it is not

controllable with all parameters and all initial values, but is

controllable with certain ranges of parameters and initial

values.

7. Conclusions

We have described the control of chaos in chaotic

neural networks. We used a method, improved to deal with

high-dimensional system control. This was based on the

method in which an unstable period point is stabilized by

exponential feedback on the system parameters. For sim-

plicity, we studied the case of a network where the chaotic

neuron is homogeneously coupled, with 1-dimensional

nearest neighbor coupling and with global coupling. We

showed that stabilization is possible even when noise is

added.

As for chaos control in a high-dimension system,

Gang and Zhilin have achieved system control with partial

element control, without controlling all of the elements, in

the coupled map lattice (CML) system [9]. Auerbach has

realized CML control using a method based on the OGY

method [10]. These results show that all elements of a

system can be controlled with partial element control using

Fig. 5. Controlling chaos to stabilize period-2 points of a neural network with 1-dimensional nearest neighbor coupling.

79

Fig. 6. Controlling chaos in a neural network with global coupling.

Fig. 7. Dependence of parameter Ki(t) , FH in coercing chaos to a period-1 point of a nearest neighbor coupling network.

80

the system cooperative dynamics and also that partial infor-

mation from the reference point can be used for coercion to

that reference point. In other words, it is self-associative

memory that restores a reference point from a part of that

reference point. It is important to develop and realize such

a method for chaotic neural networks. For neural networks

with random interaction, where weight coupling is con-

structed from the auto-correlation matrix of the memorized

patterns, determination of the unstable period points is a

prerequisite. By solving these problems, true auto-correla-

tion associative memory based on chaotic neural networks

can be achieved.

Acknowledgments. We would like to thank Dr.

Yukio Tokunaga in the Image Processing Department of the

NTT Human Interface Laboratories for his support for this

work. We would also like to thank staff members of the

Image Processing Department for their valuable advice.

REFERENCES

1. K. Aihara, T. Takabe, and M. Toyoda. Chaotic neural

networks. Phys. Lett. A, 144, pp. 333�340 (1990).

2. M. Adachi, K. Aihara, and M. Kotani. Nonlinear

associative dynamics in a chaotic neural network.

Proc. 2nd Int. Conf. Fuzzy Logic and Neural Net-

works, pp. 947�950 (1992).

3. T. Kasahara and K. Nakagawa. Parameter-control

type chaotic neuron and its application. Communica-

tions Theory (A), J78-A, No. 2, pp. 114�122 (1995).

4. E. Ott, C. Grebogi, and J.A. Yorke. Controlling chaos.

Phys. Rev. Lett., 64, pp. 1196�1199 (1990).

5. E.R. Hunt. Stabilizing high-period orbits in a chaotic

system: The diode resonator. Phys. Rev. Lett., 67, pp.

1953�1955 (1991).

6. B. Peng, V. Petrov, and K. Showalter. Controlling

low-dimensional chaos by proportional feedback.

Physica A, 188, pp. 210�216 (1992).

7. K. Pyragas. Continuous control of chaos by self-con-

trolling feedback. Phys. Lett. A, 170, pp. 421�428

(1992).

8. S.D. Gadre and V.S. Varma. Controlling chaos using

an exponential control. Preprint (1994).

9. H. Gang and Q. Zhilin. Controlling spatiotemporal

chaos in coupled map lattice systems. Phys. Rev.

Lett., 72, pp. 68�71 (1994).

10. D. Auerbach. Controlling extended systems of cha-

otic elements. Phys. Rev. Lett., 72, pp. 1184�1187

(1994).

AUTHORS (from left to right)

Shin Mizutani (member) received his B.S. degree in physics from Science University of Tokyo in 1989, and his M.S.

degree in physics from Osaka University in 1991. The same year, he joined NTT. He is currently a researcher at the NTT Human

Interface Laboratories. He has been engaged in research of neural network and visual information processing.

Takuya Sano (nonmember) received his M.S. degree in physics from Tokyo Institute of Technology in 1991. In the same

year, he joined the transmission processing department of NTT Transmission System Research Laboratories. He has been

engaged in research on optical subscription systems and optical nonlinear dynamics. He is currently a researcher in optical

subscriber system research department of the NTT Optical Network Systems Laboratories. He is a member of the Optical Society

of America and the National Geographic Society.

Tadasu Uchiyama (nonmember) received his B.S. degree and his M.S. degree in physics from Nagoya University in

1985 and 1987, respectively. In 1987, he joined NTT Human Interface Laboratories. He has been engaged in research on neural

networks and recognition of handwritten characters. He is currently a chief researcher at the same laboratories.

81

AUTHORS (continued)

Noboru Sonehara (member) received his M.S. degree from Shinshu University in 1978. In the same year, he joined the

Yokosuka Electrical Communications Laboratories of NTT, where he engaged in the development of fax machines and the

international standardization of G4 fax machines and the office document structure ODA. He began work at the Visual and

Auditory System Laboratories of the ATR International Electrical Communication Basic Technology Laboratories in 1988. He

was engaged in research on neural network image information processing, fractal image coding, and massively parallel

computing. He is currently a group leader of the image processing research group at the NTT Human Interface Laboratories.

He has an engineering doctorate.

82