publications from the thesis - shodhgangashodhganga.inflibnet.ac.in/bitstream/10603/13127/17/17...2....

Publications from the Thesis

Papers in International Journals

1. Syed Atiqur Rahman and Mohd. Samar Ansari. A Neural

Circuit with Transcendental Energy Function for Solving

System of Linear Equations. Analog Integrated Circuits and

Signal Processing, 66(3):433–440, 2011.

2. Mohd. Samar Ansari and Syed Atiqur Rahman. DVCC-

Based Non-linear Feedback Neural Circuit for Solving Sys-

tem of Linear Equations. Circuits Systems and Signal Process-

ing, 30(5):1029–1045, 2011.

3. Mohd. Samar Ansari and Syed Atiqur Rahman. Non-

linear Feedback Neural Network for Solution of Quadratic

Programming Problems. International Journal of Computer

Applications, 39(2):44–48, 2012.

4. Mohd. Samar Ansari and Syed Atiqur Rahman. A Non-

Linear Feedback Neural Network for Graph Coloring. In-

ternational Journal of Computer Applications, 39(16):31–33, 2012.

5. Mohd. Samar Ansari and Syed Atiqur Rahman. A Non-

Linear Feedback Neural Network for Solving Quadratic

Programming Problems. Journal of Active and Passive Elec-

tronic Devices, Accepted for Publication. Reference ID: RC09-53.

Book Chapter

1. S. Maheshwari, Mohd. Samar Ansari and S. A. Rahman.

Linear and Non-Linear Applications of CMOS DVCC. In

CMOS Technology, Nova Science Publishers, U.S.A., 2010.

Papers in International Conferences

1. S.A. Rahman and M.S. Ansari. Solution of Simultaneous

Linear Equations using Feedback Neural Network. In Pro-

ceedings of International Conference on Recent Applications of Soft

Computing in Engineering and Technology (RASIET-07), IET, Al-

war, India, 2007.

2. S.J. Arif, S.A. Rahman and M.S. Ansari. A NonLinear

Feedback Neural Network for Graph Coloring. In Pro-

ceedings of International Conference on Recent Applications of Soft

Computing in Engineering and Technology (RASIET-07), IET, Al-

war, India, 2007.

3. S.A. Rahman and M.S. Ansari. Solution of Quadratic Pro-

gramming Problem using Feedback Neural Network. In

Proceedings of International Conference on Intelligent Systems and

Networks (ISN-2008), Computational Intelligence Laboratory, In-

stitute of Science and Technology, Klawad, India, 2008.

4. Mohd. Samar Ansari and Syed Atiqur Rahman. A non-

linear neural circuit for solving system of simultaneous

linear equations. In Proceedings of International Conference

on Multimedia, Signal Processing and Communication Technolo-

gies (IMPACT-2009), Aligarh, 2009.

5. Mohd. Samar Ansari and Syed Atiqur Rahman. A novel

current-mode non-linear feedback neural circuit for solv-

ing linear equations. In Proceedings of International Conference

on Multimedia, Signal Processing and Communication Technologies

(IMPACT-2009), Aligarh, 2009.

6. Mohd. Samar Ansari. QPP: Solution using feedback neu-

ral network employing transconducting synaptic intercon-

nections. In Proceedings of International Conference on Intelli-

gent Systems and Nanotechnology (IISN-2010), Computational In-

telligence Laboratory, Institute of Science and Technology, Klawad,

India, 2010.

7. Mohd. Samar Ansari and Syed Atiqur Rahman. A Voltage-

Mode Analog Circuit for Solving Linear Programming Prob-

lems. In th International Conference on Computer Applications

in Electrical Engineering Recent Advances (CERA-09), IIT Roor-

kee, India, 2010.

8. Mohd. Samar Ansari and Syed Atiqur Rahman. DVCC-

based Non-Linear Feedback Neural Circuit for Solving

Quadratic Programming Problems. In th International Con-

ference on Computer Applications in Electrical Engineering Recent

Advances (CERA-09), IIT Roorkee, India, 2010.

9. Mohd. Samar Ansari and Syed Atiqur Rahman. A DVCC-

based Non-Linear Analog Circuit for solving Linear Pro-

gramming Problems. In International Conference on on Power,

Control and Embedded Systems (ICPCES-2010), MNNIT, Allahabad,

2010.

10. Mohd. Samar Ansari. Employing Differential Voltage

Current Conveyor in Graph Coloring Applications. In

International Conference on on Power, Control and Embedded Sys-

tems (ICPCES-2010), MNNIT, Allahabad, 2010.

Papers Under Review

1. Mohd. Samar Ansari and Syed Atiqur Rahman. Current-

mode Integrable Non-Linear Feedback Neural Circuit for

Solving Linear Equations. Circuits Systems and Signal Pro-

cessing

.

2. Mohd. Samar Ansari and Syed Atiqur Rahman. Current-

mode Integrable Non-Linear Feedback Neural Circuit for

Solving Linear Equations. Radioengineering

.

A neural circuit with transcendental energy function for solvingsystem of linear equations

Syed Atiqur Rahman • Mohd. Samar Ansari

Received: 16 November 2008 / Revised: 16 March 2010 / Accepted: 6 August 2010 / Published online: 21 August 2010

Springer Science+Business Media, LLC 2010

Abstract A feedback neural network based circuit to

solve a system of simultaneous linear equations is pre-

sented. The circuit has an associated transcendental energy

function that ensures fast convergence to the exact solution

while enjoying reduction in hardware complexity over

existing schemes. The proof of the energy function has

been given and it is shown that the gradient network con-

verges exactly to the solution of the system of equations.

PSPICE simulation results are presented for linear systems

of equations of various sizes and are found to agree exactly

with the algebraic solution. Hardware implementation for

small sized problems further confirm the circuit operation.

Keywords Neural network applications Neural network

hardware Nonlinear circuits Linear algebra Linear equations

1 Introduction

The solution of linear system of equations is key to many

real-time engineering applications viz. real-time speech

coding, image processing, stochastic modelling, and com-

puter-aided realistic three-dimensional image synthesis

[1–4]. The problems to be solved are often of very large

size, so that large computer resources or massive parallel

computer systems with distributed memory are needed to

solve them [1]. Massively parallel processing and fast

convergence, which are inherent to neural networks, are

utilized to reduce the time taken to arrive at the solution

[1–3]. However, existing neural networks have variable

penalty parameters which decrease to zero as time

increases to infinity in order to get better accuracy of

solution. Therefore, the time taken to arrive at the solution

in such cases is not always short [3, 4].

Hardware solutions based on neural networks have been

put forward by Xia et al. [2], Cichocki and Ubehauen [3]

and Wang [4]. To solve an n-variable system of equations,

the network of [4] uses three operational amplifiers, one

capacitor and (n ? 5) resistances to emulate a single neu-

ron and the time to arrive at the solution is of the order of

hundreds of milliseconds. The network of [3] provides a

significantly improved solution time of around a micro-

second but at the cost of increased hardware complexity.

Each neuron in [3] comprises of three weighted summers

and an inverting integrator. The architecture of [2], which

is a generalized neural network based implementation of

Censor and Elfving’s method for linear inequalities, also

uses an approach similar to [4] utilizing weighted adders

and integrators to realize the neurons.

A new architecture with fast convergence and reduced

circuit complexity is presented. The proposed architecture

uses non-linear feedback which leads to a new energy

function that involves transcendental terms. This tran-

scendental energy function is fundamentally different from

the standard quadratic form associated with Hopfield net-

work and its variations. Along with presenting the analysis

of the proposed circuit and the proof of the energy func-

tion, it is also shown that the stable state of the network

corresponds exactly with the solution of the given system

of linear equations.

S. A. Rahman M. S. Ansari (&)

Department of Electronics Engineering, Aligarh Muslim

University, Aligarh, India

e-mail: [email protected]

S. A. Rahman

e-mail: [email protected]

123

Analog Integr Circ Sig Process (2011) 66:433–440

DOI 10.1007/s10470-010-9524-2

2 Proposed neural network

Let the simultaneous linear equations to be solved are

AV ¼ B ð1Þ

where

A ¼

a11 a12 . . . a1n

a21 a22 . . . a2n

..

. ...

. . . ...

an1 an2 . . . ann

26664

37775 ð2Þ

B ¼

b1

b2

..

.

bn

26664

37775 ð3Þ

V ¼

V1

V2

..

.

Vn

26664

37775 ð4Þ

where V1, V2,…, Vn are the variables and aij and bi are

constants. We will assume that the coefficient matrix A is

invertible, and hence, the system of linear equations (1) is

consistent and not under-determined. In other words, the

linear system (1) has a uniquely determined solution.

The proposed neural-network based circuit to solve the

system of equations of (1) is presented in Fig. 1. As can be

seen from Fig. 1, individual equations from the set of

equations to be solved are passed through non-linear syn-

apses which are realized using comparators.

The output of the i-th comparator can be modelled by

xi ¼ Vm tanh b ai1V1 þ ai2V2 þ þ ainVn bið Þ ð5Þ

where b is the open-loop gain of the comparator (practi-

cally very high), ± Vm are the output voltage levels of the

comparator and V1, V2,…, Vn are the neuron outputs.

The outputs of the comparators are fed to neurons

having weighted inputs. These weighted neurons are real-

ized by using opamps where the resistors Rij act as weights.

Rpi and Cpi are the input resistance and capacitance of the

opamp corresponding to the i-th neuron. These parasitic

components are included to model the dynamic nature of

the opamp.

Node equation for node ‘Ni’ gives the equation of

motion of the i-th neuron as

Cpidui

dt¼ x1

R1iþ x2

R2iþ þ xn

Rni ui

1

Ri

ð6Þ

where ui is the internal state of the i-th neuron, and

Ri ¼ R1i k R2i k Rni k Rpi ð7Þ

Using (5) in (6) results in

Cpidui

dt¼ Vm tanh b a11V1 þ a12V2 þ þ a1nVn b1ð Þ

R1i

þ Vm tanh b a21V1 þ a22V2 þ þ a2nVn b2ð ÞR2i

þ þ Vm tanh b an1V1 þ an2V2 þ þ annVn bnð ÞRni

ui

Rið8Þ

Moreover, as has been shown in section-III, the network

in Fig. 1 can be associated with an Energy Function ‘E’

given by

E ¼ Vm

Xn

i¼1

ln cosh bXn

j¼1

aijVj bi

!Xn

i¼1

1

Ri

ZVi

0

uidV

ð9Þ

From (9), it follows that

oE

oV1

¼Vma11 tanh b a11V1 þ a12V2 þ þ a1nVn b1ð Þ

þ Vma21 tanh b a21V1 þ a22V2 þ þ a2nVn b2ð Þþ þ Vman1 tanh b an1V1 þ an2V2ð

þ þ annVn bnÞ u1

1

R1

ð10Þ

Also, if ‘E’ is the Energy Function, it must satisfy the

following condition [5].

oE

oV1

¼ KCp1

du1

dtð11Þ

where ‘K’ is a constant of proportionality and has the

dimensions of resistance.

Comparing (8) and (10) according to (11) yields

a11 ¼K

R11

; a21 ¼K

R21

; . . .; an1 ¼K

Rn1

ð12Þ

The values of the resistances of Fig. 1 can be easily

calculated by choosing a suitable value of ‘K’ and then

using (12). Analysis on similar lines can be performed to

obtain the values of the weights for the remaining neurons.

a11

V1+ a

12V

2 + ...+a

1nV

n

b1

Vi

Cpi

Rpi

R1i

Rni

x1

an1

V1+ a

n2V

2 + ...+a

nnV

n

bn x

n

Ni

Fig. 1 i-th neuron of the proposed feedback neural network circuit to

solve simultaneous linear equations in n-variables

434 Analog Integr Circ Sig Process (2011) 66:433–440

123

3 Energy function

3.1 Proof of the energy function

Differentiating the energy function of (9) w.r.t. Vi, we get

oE

oVi¼ Vma1i tanh b a11V1 þ a12V2 þ þ a1nVn b1ð Þ

þ Vma2i tanh b a21V1 þ a22V2 þ þ a2nVn b2ð Þ þ

þ Vmani tanh b an1V1 þ an2V2 þ þ annVn bnð Þ ui

Ri

ð13Þ

Comparing (8) and (10), we observed that

oE

oVi¼ KCpi

dui

dtð14Þ

where K = a1iR1i ; for all iFurther, the time derivative of the energy function is

given by

dE

dt¼XN

i¼1

oE

oVi

dVi

dt¼XN

i¼1

oE

oVi

dVi

dui

dui

dtð15Þ

Using (14) in (15) we get

dE

dt¼XN

i¼1

KCidui

dt

2dVi

duið16Þ

The transfer characteristics of the output opamp used in

Fig. 1 implements the activation function of the neuron.

With ui being the inverting terminal, it is monotonically

decreasing and it can be seen that [5],

dVi

dui 0 ð17Þ

Thereby resulting in

dE

dt 0 ð18Þ

With the equality being valid for

dui

dt¼ 0; for all i ð19Þ

Equation 18 shows that the energy function can never

increase with time which is one of the conditions for a valid

energy function. The second criterion viz. the energy

function must have a lower bound is also satisfied for the

circuit of Fig. 1 wherein it may be seen that V1, V2,…, Vn are

all bounded (as they are the outputs of opamps) amounting to

‘E’, as given in (9), having a defined lower bound.

3.2 Stable states of the network

Convergence of the network to the global minimum of the

Energy Function, which is exactly the solution of the set of

linear equations, and the fact that there are no other min-

ima, can be shown as follows.

For n variables, the second term in the energy function

expression (9) is significant only near the saturating values

of the opamp and is usually neglected [6]. The energy

function can therefore be expressed as

E ¼ Vm

Xn

i¼1

ln coshbXn

j¼1

aijVj bi

!ð20Þ

From which it follows that

oE

oV1

¼ Vma11 tanhb a11V1þa12V2þ þa1nVnb1ð Þ

þVma21 tanhb a21V1þa22V2þ þa2nVnb2ð Þþ þVman1 tanhb an1V1þan2V2þ þannVnbnð Þ

ð21Þ

oE

oV2

¼Vma12 tanhb a11V1þa12V2þþa1nVnb1ð Þ

þVma22 tanhb a21V1þa22V2þþa2nVnb2ð ÞþþVman2 tanhb an1V1þan2V2þþannVnbnð Þ

..

.

ð22Þ

oE

oVn¼Vma1n tanhb a11V1þa12V2þþa1nVnb1ð Þ

þVma2n tanhb a21V1þa22V2þþa2nVnb2ð Þþ þVmann tanhb an1V1þan2V2þþannVnbnð Þ

ð23Þ

For a stationary point, we have

oE

oV1

¼ 0

oE

oV2

¼ 0

..

.

oE

oVn¼ 0

9>>>>>>>>>>=>>>>>>>>>>;

ð24Þ

which yields,

a11 tanh b a11V1 þ a12V2 þ þ a1nVn b1ð Þþ a21 tanh b a21V1 þ a22V2 þ þ a2nVn b2ð Þ þ þ an1 tanh b an1V1 þ an2V2 þ þ annVn bnð Þ ¼ 0

ð25Þ

a12 tanh b a11V1 þ a12V2 þ þ a1nVn b1ð Þþ a22 tanh b a21V1 þ a22V2 þ þ a2nVn b2ð Þ þ þ an2 tanh b an1V1 þ an2V2 þ þ annVn bnð Þ ¼ 0

..

.

ð26Þ

Analog Integr Circ Sig Process (2011) 66:433–440 435

123

a1n tanh b a11V1 þ a12V2 þ þ a1nVn b1ð Þþ a2n tanh b a21V1 þ a22V2 þ þ a2nVn b2ð Þ þ þ ann tanh b an1V1 þ an2V2 þ þ annVn bnð Þ ¼ 0

ð27Þ

denoting

tanh b a11V1 þ a12V2 þ þ a1nVn b1ð Þ ¼ A1 ð28Þ

tanh b a21V1 þ a22V2 þ þ a2nVn b2ð Þ ¼ A2 ð29Þ

..

.

tanh b an1V1 þ an2V2 þ þ annVn bnð Þ ¼ An ð30Þ

Therefore, for a stationary point we have,

a11 a12 . . . a1n

a21 a22 . . . a2n

..

. ...

. . . ...

an1 an2 . . . ann

26664

37775

A1

A2

..

.

An

26664

37775 ¼

0

0

..

.

0

2664

3775 ð31Þ

This is a homogenous system of linear equations in

variables A1, A2,…, An. Since the coefficient matrix of

the set of Eq. 31 is the same as that of (1) which is

invertible, it follows that (31) will have a uniquely

determined solution which is the trivial solution of the

homogenous system.

Therefore,

A1

A2

..

.

An

26664

37775 ¼

0

0

..

.

0

2664

3775 ð32Þ

which results in,

a11V1 þ a12V2 þ þ a1nVn b1 ¼ 0

a21V1 þ a22V2 þ þ a2nVn b2 ¼ 0

..

.

an1V1 þ an2V2 þ þ annVn bn ¼ 0

9>>>>>=>>>>>;

ð33Þ

Thus, the energy function of the proposed neural

network has a unique stationary point which coincides

exactly with the solution of the given system of linear

equations.

This Energy Function can be visualized in 3-dimensions

for a 2—variable problem and is shown in Fig. 2. From the

plot, it can be seen that there exists only one minimum to

which the network must converge.

4 PSPICE simulation results

The proposed circuit was tested using PSPICE simulation

program for solving sets of 2, 3, 4, 5, 10 and 20 simulta-

neous linear equations. The application of the proposed

circuit to a chosen 3—variable problem (34) is presented

below.

2 1 1

3 2 1

1 1 2

24

35

V1

V2

V3

24

35 ¼

5

10

6

24

35 ð34Þ

The circuit to solve (34) as obtained from Fig. 1 is

presented in Fig. 3. As can be seen, some additional

circuitry is needed to generate the inputs to the non-linear

synapses. Routine analysis yields the following values of

the resistors at the input of the non-linear synapses.

Fig. 2 Typical energy function

plot for a 2—variable problem


123

Re11 = 2.5 K, Re12 = Re13 = Re14 = 5 K, Re21 =

3.33 K, Re22 = 5 K, Re23 = 10 K, Re24 = 2.5 K,

Re31 = Re32 = 6 K, Re33 = Re34 = 3 K, R11 = R22 =

R33 =5 K, R12 = R13 = R23 = R31 = R32 = 10 K and

R21 = 3.33 K.

Algebraic analysis of (34) gives the solution as V1 =

-0.5 V, V2 = 5.5 V and V3 = 0.5 V. The results of

PSPICE simulation of the circuit of Fig. 3, shown in Fig. 5,

are found to match perfectly with the algebraic solution.

The initial node voltages were kept as V(1) = 10 mV,

V(2) = -1 mV and V(3) = 20 mV. For the purpose of

this simulation, the LM318 opamp from the Orcad library

in PSPICE was utilised. The transfer characteristics of the

opamp are presented in Fig. 4 below. The value of ‘b’ for

this opamp was measured to be 1.1 9 104 using SPICE

simulation.

The proposed circuit was further tested in PSPICE,

using other opamp models from Orcad library as well, for

solving systems of linear equations in 2, 4, 5, 10 and 20

variables. Results of simulation runs for these problems are

presented in Table 1. Each of the simulations was run using

various initial conditions in the millivolt range. As can be

seen from Table 1, the proposed network always converges

to the solution of the given system of linear equations.

5 Hardware implementation

Breadboad implementation of the proposed circuit was also

carried out. Apart from verification of the working of the

proposed circuit, the actual circuit realization also served

the purpose of testing the convergence of the circuit to the

solution starting from different initial conditions. The noise

present in any electronic circuit acts as a random initial

condition for the convergence of the neural circuit. Stan-

dard laboratory components viz. the lA741 opamp and

R11

R21

V1

a11V1+ a12V2 + a13V3

x1

u1

Rp1 Cp1

R22

R12

V2

b2

x2

u2

Rp2 Cp2

R33

R23

V3

b3

x3

u3

Rp3 Cp3

a21V1+ a22V2 + a23V3

a31V1+ a32V2 + a33V3

b1

R31

R32

R13

Re11

Re12

Re13

Re14

Re24

Re34

Re21

Re31

Re22

Re32

Re23

Re33

V1

V1

V1

V2

V2

V2

V3

V3

V3

N1

N2

N3

Fig. 3 The proposed circuit

applied to a 3—variable

problem

VIN

-1.0V -0.8V -0.6V -0.4V -0.2V -0.0V 0.2V 0.4V 0.6V 0.8V 1.0V

V(2)

-20V

-10V

0V

10V

20VFig. 4 Transfer characteristics

of the opamps used to realize

the neurons of Fig. 3


123

resistances were used for the purpose. Circuits to solve sets

of equations in 2 and 3 variables were assembled and each

circuit was tested for 25 test runs. The results of hardware

implementation are presented in Table 2.

6 Discussion

This section deals with the monolithic implementation

issues of the proposed circuit. The PSPICE simulations

assumed that all operational amplifiers are identical, and

therefore, it is required to determine how deviations from

this assumption affect the performance of the network.

Effects of variations in component values from one neuron

to another were also investigated using Monte-Carlo

analysis in PSPICE. A 10% tolerance with Gaussian

deviation profile was put on the resistances used in the

circuit of Fig. 3. The analysis was carried out for 100 runs

and the Mean Deviation was found out to be 0.0497 and

Mean Sigma (Standard Deviation) was 0.1243. Offset

analysis was also carried out by incorporating random

offset voltages (in the range of 1–10 mV) to the opamps.

The Mean Deviation in this case was measured to be

-0.0512 and the Mean Sigma (Standard Deviation) was

0.172. As can be seen, the effects of mismatches and off-

sets on the overall precision of the final results are in an

acceptable range.

In fact, the drawbacks of the use of opamps in the

proposed circuit suggest that a real, large scale imple-

mentation for solving a system of equations in several

variables might be quite different. Alternative realizations

based on the differential Eq. 5 governing the system of

neurons are being investigated. An implementation based

on operational transconductance amplifiers (OTAs) would

appear to be one viable alternative; similarly, circuits using

MOS transistors operating in the subthreshold regime

would be another. In the latter case, the tanh(.) nonlinearity

is easily obtained because of the transistor’s current–volt-

age characteristic [7] and is expected to be the focus of

future work. However, the present manuscript focuses

mainly on the principle of such a network, and a VLSI

implementation is beyond the scope of this paper.

Further, scaling issues for the proposed circuit were also

investigated. For this purpose, two problems of 10 and 20

variables each were selected. These appear in Table 1 and

were solved using the network of Fig. 1. PSPICE simula-

tions was performed and the obtained simulation results

for the 10-variable problem were V1 = -1.9999, V2 =

1.0000, V3 = 3.0001, V4 = -1.0000, V5 = 1.9999, V6 =

7.0003, V7 = -3.0000, V8 = 3.9999, V9 = -5.0000 and

V10 = -4.0002 which agree very closely with the exact

mathematical solution.

7 Conclusions

In this paper we have described a novel approach to solve

n simultaneous linear equations in n variables, which uses

n neurons and n synapses. Each neuron requires one opamp

and each synapse is implemented using one comparator.

This results in significant reduction in hardware over the

existing schemes [1–4]. From ULSI implementation point

of view, more efficient designs are being explored in order

to reduce the circuitry required for each synapse.

Hardware implementations were carried out for 2 and 3

variable problems. The results were found to match with the

mathematical solution. The working of the proposed net-

work was also verified using PSPICE for various sample

problem sets of 2–20 simultaneous linear equations. While

the simulation results confirm the validity of the approach,

issues such as the network response to a system of equations

which do not have a unique solution are yet to be explored. It

may be mentioned that the technique can be extended to the

standard linear and quadratic programming problems.

Time

0s 5us 10us 15us 20us 25us 30us 35us 40us 45us 50us

V(1) V(2) V(3)

-2.0V

0V

2.0V

4.0V

6.0VFig. 5 Simulation results for

the chosen 3—variable problem


123

Table 1 PSPICE simulation results for the proposed circuit applied to various problems


123

Acknowledgements The authors are grateful to the anonymous

reviewers for their useful suggestions, which helped to improve the paper.

References

1. Jiang, D. (2004). Analog computing for real-time solution of time-

varying linear equations. In Proceedings of ICCCAS, Vol. 2,

pp. 27–29.

2. Xia, Y., Wang, J., & Hung, D. L. (1999). Recurrent neural

networks for solving linear inequalities and equations. IEEECircuits and Systems Society: I, 46(4), 452.

3. Cichocki, A., & Unbehauen, R. (1992). Neural networks for

solving systems of linear equations and related problems. IEEECircuits and Systems Society: I, 39(2), 124.

4. Wang, J. (1992). Electronic realization of recurrent neural network

for solving simultaneous linear equations. Electronics Letters,28(5), 493.

5. Rahman, S. A., Jayadeva, & Duttaroy, S. C. (1999). Neural

network approach to graph colouring. Electronics Letters, 35(14),

1173–1175.

6. Tank, D. W., & Hopfield, J. J. (1986). Simple neural optimization

networks: An A/D converter, signal decision network, and linear

programming circuit. IEEE Transactions on Circuits and SystemsCAS, 33(5), 533.

7. Newcomb, R. W., & Lohn, J. D. (1998) Analog VLSI for neural

networks. In The handbook of brain theory and neural networks.

Cambridge, MA: The MIT Press.

Syed Atiqur Rahman did

B.Sc.(Eng.) in Electrical Engi-

neering in 1988, and M.Sc.(Eng.)

in Electronics and Communica-

tion in 1994, from Aligarh

Muslim University, Aligarh. He

was appointed as Lecturer in

the Department of Electronics

Engineering at AMU, Aligarh in

1988. He joined as Reader in

Computer Engineering and then

Electronics Engineering in the

same University in 1997, and is

continuing at the latter post. His

fields of interest are Electronic

Circuits and Artificial Neural Networks.

Mohd. Samar Ansari received

B.Tech. degree in Electronics

Engineering from the Aligarh

Muslim University, Aligarh,

India, in 2001 and M.Tech.

degree in Electronics Engineer-

ing, with specialization in elec-

tronic circuit and system design,

in 2007. He is currently a

Lecturer in the Department of

Electronics Engineering, Ali-

garh Muslim University, where

he teaches electronic devices &

circuits and microelectronics.

His research interests include

micro-electronics, linear integrated circuit applications, neural

networks, and semiconductor physics.

Table 2 Hardware test results of the proposed circuit applied to 2 and 3 variable problems

Chosen set of

linear equations

Algebraic

solution

Hardware test results

(first five runs)

Standard deviation,

r (for 25 runs)

1 1

1 3

V1

V2

¼ 10

10

V1

V2

¼ 10

0

9:89

0:18

" #9:90

0:18

" #9:89

0:18

" #9:89

0:18

" #9:89

0:18

" #r1

r2

¼ 0:039

0:017

1 1 3

1 3 1

3 1 1

24

35

V1

V2

V3

24

35 ¼

10

10

10

24

35

V1

V2

V3

24

35 ¼

2

2

2

2435

2:04

1:98

2:00

264

375

2:01

1:97

2:00

264

375

2:04

1:98

2:01

264

375

2:04

1:98

2:00

264

375

2:04

1:97

2:01

264

375

r1

r2

r3

24

35 ¼

0:044

0:025

0:032

24

35


123

Circuits Syst Signal Process (2011) 30:1029–1045DOI 10.1007/s00034-010-9261-x

DVCC-Based Non-linear Feedback Neural Circuitfor Solving System of Linear Equations

M. Samar Ansari · Syed Atiqur Rahman

Received: 26 October 2009 / Revised: 20 April 2010 / Published online: 14 January 2011© Springer Science+Business Media, LLC 2011

Abstract A neural circuit to solve a system of simultaneous linear equations is pre-sented. The circuit employs non-linear feedback to achieve a transcendental energyfunction that ensures fast convergence to the exact solution while enjoying reduc-tion in hardware complexity over existing schemes. A new building block for analogsignal processing, the digitally controlled differential voltage current conveyor (DC-DVCC) is introduced and is utilized for the non-linear synaptic interconnections be-tween neurons. The proof of the energy function has been given and it is shown thatthe gradient network converges exactly to the solution of the system of equations.PSPICE simulation results are presented for linear systems of equations of varioussizes and are found to be in close agreement with the algebraic solution. The use ofCMOS DC-DVCCs and operational amplifiers facilitates monolithic integration.

Keywords Neural network applications · Neural network hardware · Non-linearcircuits · Linear algebra · Linear equations · Digitally Controlled DVCC

1 Introduction

Solution of a system of simultaneous linear equations has been a primary goal forcomputation since the time of the abacus. References to word problems requiring thesolution of a system of linear equations appear in ancient Babylonian texts datingback to circa 300 BC [3]. Some examples of applications requiring the solution ofsystems of linear equations are curve fitting, electrical circuit analysis, multiple cor-

M.S. Ansari () · S.A. RahmanDepartment of Electronics Engineering, Aligarh Muslim University, Aligarh, Indiae-mail: [email protected]

S.A. Rahmane-mail: [email protected]

mailto:[email protected]

mailto:[email protected]

1030 Circuits Syst Signal Process (2011) 30:1029–1045

relation as well as real-time applications like real-time speech coding, image process-ing, stochastic modelling, and computer-aided realistic three-dimensional image syn-thesis [1, 3, 5, 10, 11]. Traditional methods for solving systems of linear equationstypically involve an iterative process [3], but long computational time limits their us-age. There is an alternative approach to the solution of this problem. It is to exploit theartificial neural networks (ANN’s), which can be considered as an analog computerrelying on a highly simplified model of neurons [1, 5, 10, 11]. Massively parallelprocessing and fast convergence, which are inherent to neural networks, are utilizedto reduce the time taken to arrive at the solution [1, 10, 11]. However, existing neuralnetworks have variable penalty parameters which decrease to zero as time increasesto infinity in order to get better accuracy of solution. Therefore, the time taken toarrive at the solution in such cases is not always short [1, 10].

Hardware solutions for solving a system of linear equations based on neural net-works have been put forward by Xia, Wang and Hung [11], Cichocki and Unbehauen[1] and Wang [10]. To solve an n-variable system of equations, the network of [10]uses three operational amplifiers, one capacitor and (n + 5) resistances to emulatea single neuron and the time to arrive at the solution is of the order of hundreds ofmilliseconds. The network of [1] provides a significantly improved solution time ofaround a microsecond but at the cost of increased hardware complexity. Each neuronin [1] comprises three weighted summers and an inverting integrator. The architec-ture of [11], which is a generalized neural network based implementation of Censorand Elfving’s method for linear inequalities, also uses an approach similar to [10],utilizing weighted adders and integrators to realize the neurons.

A new digital programmable architecture with fast convergence and reduced cir-cuit complexity is presented. The proposed architecture uses non-linear feedbackwhich leads to a new energy function that involves transcendental terms. This tran-scendental energy function is fundamentally different from the standard quadraticform associated with Hopfield network and its variations. The non-linear feedbackis realized by utilizing a new building block viz. the digitally controlled differen-tial voltage current conveyor (DC-DVCC). Along with presenting the analysis of theproposed circuit and the proof of the energy function, it is also shown that the stablestate of the network corresponds exactly with the solution of the given system of lin-ear equations. Further, the use of CMOS DC-DVCCs and opamps makes monolithicintegration viable.

This paper is organized as follows. A new digitally programmable analog build-ing block (DC-DVCC) is presented in Sect. 2. Section 3 contains the details of theproposed network along with the design equations. Proofs of the energy function andvalidity of the solution are given in Sect. 4. Section 5 presents the results of SPICEsimulation of the circuit applied to solve various sample equation sets. A discussionon VLSI implementability of the proposed circuit appears in Sect. 5. Some conclusiveremarks appear in Sect. 6.

2 Digitally Controlled DVCC

The differential voltage current conveyor (DVCC) was proposed in 1997 [2] as a fiveterminal device characterized by the following port relations (Fig. 1):

Circuits Syst Signal Process (2011) 30:1029–1045 1031

Fig. 1 Block representation ofthe DVCC

Fig. 2 CMOS implementation of DVCC

⎡⎢⎢⎢⎢⎢⎢⎣

VX

IY1

IY2

I+Z

I−Z

⎤⎥⎥⎥⎥⎥⎥⎦

=

⎡⎢⎢⎢⎢⎣

0 1 −1 0 00 0 0 0 00 0 0 0 0k 0 0 0 0k 0 0 0 0

⎤⎥⎥⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎣

IX

VY1

VY2

V +Z

V −Z

⎤⎥⎥⎥⎥⎥⎥⎦

(1)

While the voltage on the X-terminal follows the difference in voltages of terminalsY1 and Y2, a current injected at the X-terminal is replicated by a factor k to the Z-terminals. For the Z+ terminal, the direction of the conveyed current is the same asthat of the current flowing in the X-terminal whereas for the Z− terminal, the currentflows in the opposite direction. Ideally, k is unity.

One possible CMOS realization of the DVCC is shown in Fig. 2 [6]. As can beseen, the current at Z+ port will be the same as the current in the X terminal and thecurrent at the Z− terminal will have the same magnitude but opposite direction as theX port current.

Although both Z+ and Z− types of current outputs are mentioned in (1), theDVCCs used in the proposed network use only Z+ type of outputs. Therefore, wenext describe the proposed digitally controlled DVCC with only Z+ outputs. Thetechnique is to control the current transfer gain parameter k of the DVCC by replac-ing the Z terminal transistors of the DVCC with transistor arrays associated with


Fig. 3 CMOS realization of Digitally Controlled DVCC with gain k

switches [4]. The gain parameter k can take values from 1 to (2n − 1), where n rep-resents the number of transistor arrays. Actually, the transistor arrays implement acurrent summing network (CSN) at the Z terminal. The circuit of DC-DVCC ob-tained after suitable modifications in the circuit of Fig. 2 is presented in Fig. 3.

The CSN consists of n transistor pairs, whose NMOS and PMOS aspect ratios aregiven by:

NMOS:

(W

L

)

i

= 2i

(W

L

)

11, i = 0,1,2, . . . , (n − 1) (2)

PMOS:

(W

L

)

i

= 2i

(W

L

)

7, i = 0,1,2, . . . , (n − 1) (3)

Therefore, the current at the Z terminal, assumed flowing out of the DC-DVCC, canbe expressed by

IZ =n−1∑i=0

di2i (I7 − I11) (4)

Therefore, the proposed DC-DVCC provides a current transfer gain equal to

k = IZ

IX

=∑n−1

i=0 di2i (I7 − I11)

(I7 − I11)=

n−1∑i=0

di2i (5)

Parameter di represents the digital code-bit applied to the ith branch in the CSN.Depending upon its value, it enables or disables the current to flow in that particularbranch. It is instructive to note the numbering of the transistors in the CSN. Transis-tors labeled M8(i) and M12(i) refer to the PMOS and NMOS transistors in the CSNthat have been put there in the place of their counterparts of Fig. 2. Transistors MD8(i)

and MD12(i) are the actual digital control transistors as the digital control bits d0, d1and d2 are applied at their respective gate terminals.


For the DC-DVCC, comparator action can be achieved by putting RX → 0 i.e. bydirectly grounding the X-terminal [6]. For such a case, the current in the X-port willsaturate and can be written as

IX = Im tanhβ(VY1 − VY2) (6)

where β is the open-loop gain of the comparator (practically very high), ±Im are thesaturated output current levels of the comparator and V1,V2, . . . , Vn are the neuronoutputs. Equation (6) can also be written in an equivalent notation as

IX = gVm tanhβ(VY1 − VY2) (7)

where g is the transconductance from the input voltage ports (Y1 and Y2) to the outputcurrent port X and is governed by the resistance at the X port; and ±Vm are thebiasing voltages of the DVCC-based comparator.

By virtue of DVCC action, this current will be transferred to the Z+ ports as

I+Z = kIX = kgVm tanhβ(VY1 − VY2) (8)

Therefore, the current at the Z terminal can be made a digitally controlled scaledreplica of the X port current. This property will be utilized in the design of the multi-output DVCCs needed for the proposed circuit. It may be mentioned that multiple Z+outputs can be obtained by repeating the output stage comprising transistors M8(i)

and M12(i).

3 Proposed Neural Network

Let the simultaneous linear equations to be solved are

AV = B (9)

where

A =

⎡⎢⎢⎣

a11 a12 . . . a1n

a21 a22 . . . a2n...

... . . ....

an1 an2 . . . ann

⎤⎥⎥⎦ (10)

B =

⎡⎢⎢⎣

b1b2...

bn

⎤⎥⎥⎦ (11)

V =

⎡⎢⎢⎣

V1V2...

Vn

⎤⎥⎥⎦ (12)


Fig. 4 First neuron of the proposed digitally programmable, feedback neural circuit to solve simultaneouslinear equations in n-variables

where V1,V2, . . . , Vn are the variables and aij and bi are constants. We will assumethat the coefficient matrix A is invertible, and hence, the system of linear equations(9) is consistent and not under-determined. In other words, the linear system (9) hasa uniquely determined solution.

The proposed neural network based circuit to solve the system of equations of (9)is presented in Fig. 4. As can be seen from Fig. 4, individual equations from the setof equations to be solved are passed through non-linear synapses which are realizedusing multi-output DC-DVCC based comparators.

The outputs of the comparators are fed to neurons having weighted inputs. Theseweighted neurons are realized by using opamps where the scaled currents comingfrom various comparators act as weights. Rpi and Cpi are the input resistance andcapacitance of the opamp corresponding to the ith neuron. These parasitic compo-nents are included to model the dynamic nature of the opamp.

Node equation for node ‘Ni ’ gives the equation of motion of the ith neuron as

Cpi

dui

dt= k1iIX1 + k2iIX2 + · · · + kniIXn − ui

[1

Rpi

](13)

where ui is the internal state of the ith neuron and kji is the current scaling factor atthe j th output of ith DVCC. Using (7) in (13) results in

Cpi

dui

dt= k1igVm tanhβ(a11V1 + a12V2 + · · · + a1nVn − b1)

+ k2igVm tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·+ knigVm tanhβ(an1V1 + an2V2 + · · · + annVn − bn) − ui

Rpi

(14)


Moreover, as has been shown in Sect. 4, the network in Fig. 4 can be associated withan Energy Function ‘E’ given by

E = Vm

n∑i=1

ln coshβ

(n∑

j=1

aijVj − bi

)−

n∑i=1

1

Ri

∫ Vi

0ui dVi (15)


∂E

∂V1= Vma11 tanhβ(a11V1 + a12V2 + · · · + a1nVn − b1)

+ Vma21 tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·

+ Vman1 tanhβ(an1V1 + an2V2 + · · · + annVn − bn) − u1

[1

Rp1

](16)

Also, if ‘E’ is the energy function, it must satisfy the following condition [8].

∂E

∂Vi

= KRCpi

dui

dt(17)

where KR is a constant of proportionality having the dimensions of resistance and isnormalized to (1/g) for simplicity.


⎡⎢⎢⎣

k11 k12 . . . k1n

k21 k22 . . . k2n...

... . . ....

kn1 kn2 . . . knn

⎤⎥⎥⎦ =

⎡⎢⎢⎣

a11 a12 . . . a1n

a21 a22 . . . a2n...

... . . ....

an1 an2 . . . ann

⎤⎥⎥⎦ (18)

4 Energy Function

4.1 Proof of the Energy Function

Differentiating the energy function of (15) w.r.t. Vi , we get

∂E

∂Vi

= Vma1i tanhβ(a11V1 + a12V2 + · · · + a1nVn − b1)

+ Vma2i tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·+ Vmani tanhβ(an1V1 + an2V2 + · · · + annVn − bn) − ui

Rpi

(19)

Further, the time derivative of the energy function is given by

dE

dt=

N∑i=1

∂E

∂Vi

dVi

dt=

N∑i=1

∂E

∂Vi

dVi

dui

dui

dt(20)



dE

dt=

N∑i=1

KCi

(dui

dt

)2dVi

dui

(21)

The transfer characteristics of the output opamp used in Fig. 4 implements the activa-tion function of the neuron. With ui being the inverting terminal, it is monotonicallydecreasing and it can be seen that [8]

dVi

dui

≤ 0 (22)

thereby resulting in

dE

dt≤ 0 (23)

with the equality being valid for

dui

dt= 0; for all i (24)

Equation (23) shows that the energy function can never increase with time which isone of the conditions for a valid energy function. The second criterion viz. the energyfunction must have a lower bound is also satisfied for the circuit of Fig. 4 wherein itmay be seen that V1,V2, . . . , Vn are all bounded (as they are the outputs of opamps)amounting to ‘E’, as given in (15), having a defined lower bound.

4.2 Stable States of the Network

Convergence of the network to the global minimum of the energy function, which isexactly the solution of the set of linear equations, and the fact that there are no otherminima, can be shown as follows.

For n variables, the second term in the energy function expression (15) is signif-icant only near the saturating values of the opamp and is usually neglected [9]. Theenergy function can therefore be expressed as

E = Vm

n∑i=1

ln coshβ

(n∑

j=1

aijVj − bi

)(25)

from which it follows that

∂E


+ Vma21 tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·+ Vman1 tanhβ(an1V1 + an2V2 + · · · + annVn − bn) (26)


∂E


+ Vma22 tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·+ Vman2 tanhβ(an1V1 + an2V2 + · · · + annVn − bn) (27)

...

∂E

∂Vn

= Vma1n tanhβ(a11V1 + a12V2 + · · · + a1nVn − b1)

+ Vma2n tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·+ Vmann tanhβ(an1V1 + an2V2 + · · · + annVn − bn) (28)

For a stationary point, we have

∂E

∂V1= 0

∂E

∂V2= 0

...

∂E

∂Vn

= 0

⎫⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎭

(29)

which yields

a11 tanhβ(a11V1 + a12V2 + · · · + a1nVn − b1)

+ a21 tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·+ an1 tanhβ(an1V1 + an2V2 + · · · + annVn − bn) = 0 (30)

a12 tanhβ(a11V1 + a12V2 + · · · + a1nVn − b1)

+ a22 tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·+ an2 tanhβ(an1V1 + an2V2 + · · · + annVn − bn) = 0 (31)

...

a1n tanhβ(a11V1 + a12V2 + · · · + a1nVn − b1)

+ a2n tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) + · · ·+ ann tanhβ(an1V1 + an2V2 + · · · + annVn − bn) = 0 (32)

Denoting

tanhβ(a11V1 + a12V2 + · · · + a1nVn − b1) = A1 (33)

tanhβ(a21V1 + a22V2 + · · · + a2nVn − b2) = A2 (34)...

tanhβ(an1V1 + an2V2 + · · · + annVn − bn) = An (35)


Therefore, for a stationary point we have⎡⎢⎢⎣

a11 a12 . . . a1n

a21 a22 . . . a2n...

... . . ....

an1 an2 . . . ann

⎤⎥⎥⎦

⎡⎢⎢⎣

A1A2...

An

⎤⎥⎥⎦ =

⎡⎢⎢⎣

00...0

⎤⎥⎥⎦ (36)

This is a homogeneous system of linear equations in variables A1,A2, . . . ,An. Sincethe coefficient matrix of the set of (36) is the same as that of (9) which is invertible, itfollows that (36) will have a uniquely determined solution which is the trivial solutionof the homogeneous system.

Therefore, ⎡⎢⎢⎣

A1A2...

An

⎤⎥⎥⎦ =

⎡⎢⎢⎣

00...0

⎤⎥⎥⎦ (37)

which results ina11V1 + a12V2 + · · · + a1nVn − b1 = 0

a21V1 + a22V2 + · · · + a2nVn − b2 = 0...

an1V1 + an2V2 + · · · + annVn − bn = 0

⎫⎪⎪⎪⎬⎪⎪⎪⎭

(38)

Thus, the energy function of the proposed neural network has a unique stationarypoint which coincides exactly with the solution of the given system of linear equa-tions.

This energy function can be visualized in three dimensions for a two-variableproblem and is shown in Fig. 5. From the plot, it can be seen that there exists onlyone minimum, to which the network must converge.

Fig. 5 Typical energy function plot for a two-variable problem


5 PSPICE Simulation Results

The proposed circuit was tested using PSPICE simulation program for solving setsof two, three, four, five and 10 simultaneous linear equations. The application of theproposed circuit to a chosen three-variable problem (39) is presented below.

⎡⎣

2 1 13 2 11 1 2

⎤⎦

⎡⎣

V1V2V3

⎤⎦ =

⎡⎣

5106

⎤⎦ (39)

The circuit to solve (39) as obtained from Fig. 4 is presented in Fig. 6. The values ofthe current scaling coefficients kji are given as

⎡⎣

k11 k12 k13k21 k22 k23k31 k32 k33

⎤⎦ =

⎡⎣

2 1 13 2 11 1 2

⎤⎦ (40)

For implementing these values of kji appropriate values of the control words wereselected. For the first DC-DVCC, the three control words were kept as [0 1 0], [0 0 1]

Fig. 6 The proposed circuit applied to a three-variable problem


Fig. 7 Simulation results for the chosen three-variable problem

and [0 0 1]. Similarly, the control words for the second DC-DVCC were [0 1 1],[0 1 0] and [0 0 1]. Finally, for the third DC-DVCC the control words were [0 0 1],[0 0 1] and [0 1 0]. Thus, a proper selection of the control words can be used toachieve any current scaling factor between 1 and 7 and therefore this particular re-alization is restricted to solving systems of equations with coefficients in that range.However, that range may be widened further by having four control bits in the CSN.

As can be seen, some additional circuitry is needed to generate the inputs to thenon-linear synapses. Routine analysis yields the following values of the resistors atthe input of the non-linear synapses.

Re11 = 2.5 K, Re12 = Re13 = Re14 = 5 K, Re21 = 3.33 K, Re22 = 5 K,

Re23 = 10 K, Re24 = 2.5 K, Re31 = Re32 = 6 K, Re33 = Re34 = 3 K,

Rc11 = Rc21 = Rc31 = 1 K, Rc12 = 4 K, Rc22 = 9 K, Rc32 = 5 K.

Algebraic analysis of (39) gives the solution as V1 = −0.5 V, V2 = 5.5 V andV3 = 0.5 V. The results of PSPICE simulation of the circuit of Fig. 6, shown inFig. 7, are found to match perfectly with the algebraic solution. The initial nodevoltages were kept as V (1) = 10 mV, V (2) = −10 mV and V (3) = 20 mV. The cir-cuit for DVCC was taken from [6] and standard 0.5 micron CMOS parameters wereused for simulation purposes. The DVCCs were biased with ±2.5 V supplies. Forthe opamp, use was made of the LMC7101A CMOS opamp from National Semicon-ductor. The sub-circuit file for this opamp is available in Orcad Model Library. The


Fig. 8 Transfer characteristics of the DC-DVCC used to realize the comparator

biasing voltages for the opamps were taken to be ±15 V. The transfer characteris-tics of the DVCC-based comparator, shown in Fig. 8, were also plotted using SPICEsimulation and value of ‘β’ was found to be 2.4 × 103.

The proposed circuit was further tested in PSPICE for solving systems of linearequations in two, four, five, and ten variables. Results of simulation runs for theseproblems are presented in Table 1. Each of the simulations was run using variousinitial conditions in the millivolt range. As can be seen from Table 1, the proposednetwork always converges to the solution of the given system of linear equations.As can be seen, the obtained results are quite near the algebraic solutions with themaximum error being approximately 6% and the average error being 0.49%.

6 Performance Evaluation

This section deals with the effect of component mismatches and device non-idealitieson the accuracy of the solution obtained from the circuit proposed.

If the proposed circuit is implemented using discrete resistances, variations in thevalues of the resistances need to be considered keeping in mind the tolerances asso-ciated with discrete resistors. However, if the proposed network is targeted for mono-lithic integration, random variations in the resistance values need not be considered.In that case, all the resistances are expected to deviate from their assigned values bythe same factor [7]. Therefore, a worthy performance appraisal of the circuit couldbe obtained by testing the circuit with all resistances having the same percentage de-viation from their assigned values. Such an assessment of the quality of the obtainedsolution is presented in Table 2 from which it can be seen that the percentage error


Table 1 PSPICE Simulation results for the proposed circuit applied to different systems of linear equa-tions

[A] [B] AlgebraicSolution

Simulated Results(Using PSPICE)

Percentage Errorin the solution (%)

[V] [V]

[1 22 1

] [3.54

] [1.51

] [1.511.02

] [ −0.66−2.00

]

[2 1 13 2 11 1 1

] [5106

] [−0.55.50.5

] [−0.525.540.48

] [−4.00−0.724.00

]

⎡⎣

2 3 6 13 2 5 42 5 5 22 4 2 5

⎤⎦

⎡⎣

48.7567.554

59.25

⎤⎦

⎡⎣

3.51.55

7.25

⎤⎦

⎡⎣

3.511.535.097.19

⎤⎦

⎡⎣

−0.28−2.00−1.800.83

⎤⎦

⎡⎢⎢⎣

2 3 9 2 52 6 9 9 52 6 2 4 52 4 7 8 35 3 6 3 5

⎤⎥⎥⎦

⎡⎢⎢⎣

−54.9−91.0−54.9−74.8−70.0

⎤⎥⎥⎦

⎡⎢⎢⎣

−6.7−4.3−2.8−3.30.64

⎤⎥⎥⎦

⎡⎢⎢⎣

−6.65−4.33−2.63−3.190.63

⎤⎥⎥⎦

⎡⎢⎢⎣

0.75−0.696.073.331.56

⎤⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

1 2 1 3 4 2 1 1 1 22 1 1 3 1 2 2 1 2 31 1 4 3 3 1 1 4 4 14 2 1 5 3 3 1 1 2 21 1 5 1 2 1 2 2 5 23 3 1 2 1 1 5 2 1 15 5 1 4 1 1 3 4 2 21 1 1 2 2 2 3 3 4 14 2 2 1 3 2 5 4 3 21 2 3 1 2 3 1 2 3 4

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

10−11102

−7−9−8−3−37

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

−213

−127

−34

−5−4

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

−1.911.043.05

−0.951.947.04

−3.063.99

−4.91−3.95

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

4.50−4.00−1.665.003.00

−0.57−2.000.251.801.25

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

Table 2 Effect of variation inresistances on the obtainedresults

Percentage Variationin Resistances

Simulated Results(Using PSPICE)[V]

Percentage Error inthe Solution (%)

+2%

[−0.5085.6020.499

] [ +1.6+1.85−0.2

]

+5%

[−0.5095.6190.501

] [ +1.8+2.16+0.2

]

+10%

[−0.5145.6810.507

] [ +2.8+3.29+1.4

]

−2%

[−0.5075.6010.505

] [ +1.4+1.83+1

]

−5%

[−0.5185.6790.506

] [ +3.6+3.25+1.2

]

−10%

[−0.5165.6780.504

] [ +3.2+3.23+0.8

]


Table 3 Effect of gains of theDVCC-based comparators onthe solution quality

X-terminal resistancesof the DVCCs ()


Percentage Errorin the solution (%)

5

[−0.5125.6100.500

] [2.42.00.0

]

10

[−0.5145.6170.510

] [2.82.1

2.00

]

20

[−0.5205.6200.511

] [−4.002.182.2

]

in the solution varies with the percentage errors specified in the values of the resis-tances and even for ±10% variation in the values of the resistors, the solution pointchanges by approximately 3%.

Further, the gains of the DVCC-based comparators were varied to investigate theireffect on the solution quality. Toward that end, a resistance was connected betweenthe X-terminal of each DVCC and ground. Ideally, these resistances were assumed tohave zero value. However, by assigning different values to these resistances, the gainsof the DVCCs were varied. The solutions, as obtained for the three-variable problemof Fig. 6, are presented in Table 3. It can be seen that small variations in the gains ofthe comparators do not affect the quality of the solution.

Next, the effect of offset voltages in the DVCC-based comparators was explored.Offset voltages were applied at the Y2 inputs of the DVCCs of Fig. 6 and the results ofPSPICE simulations were compared with the algebraic solution as given in Table 4.As can be seen, the offset voltages of the comparators do not affect the obtainedsolutions to any appreciable extent. However, the error does tend to increase withincreasing offset voltages.

Finally, offset voltages for the opamps were also considered. Offset voltages wereapplied at the non-inverting inputs of the opamps of Fig. 6 and the results of PSPICEsimulations were compared with the algebraic solution as given in Table 5. As can beseen, the offset voltages of the opamps have little effect on the obtained solutions.

7 Conclusion

In this paper we have described a novel approach to solve n simultaneous linearequations in n variables, which uses n neurons and n synapses. Each neuron requiresone opamp and each synapse is implemented using one comparator. The compara-tors were realized using a new digitally controlled DVCC. This results in significantreduction in hardware over the existing schemes [1, 5, 10, 11]. From VLSI imple-mentation point of view, use of CMOS DC-DVCCs and opamps facilitates viabilityfor monolithic integration.

The working of the proposed network was verified using PSPICE for various sam-ple problem sets of two to 10 simultaneous linear equations. While the simulation


Table 4 Effect of offsetvoltages of the DVCC-basedcomparators on the solutionquality

Offset Voltage appliedat Y2 input of the DVCCs(mV)


Percentage Errorin the Solution (%)

−5

[−0.4985.6150.497

] [ −0.42.09−0.6

]

−10

[−0.5055.6200.506

] [1.02.181.2

]

−15

[−0.5125.6310.511

] [2.42.382.2

]

+5

[−0.4965.6110.494

] [ −0.82.01−1.2

]

+10

[−0.5035.6270.508

] [0.62.301.6

]

+15

[−0.5195.6290.510

] [3.82.34

2

]

Table 5 Effect of offsetvoltages of opamps on solutionquality

Offset Voltage appliedat the non-inverting inputof opamps (mV)


Percentage Error inthe Solution (%)

−5

[−0.4945.6170.507

] [−1.22.121.4

]

−10

[−0.5045.5810.510

] [0.81.47

2

]

−15

[−0.5145.5430.491

] [2.80.78−1.8

]

+5

[−0.4995.6420.497

] [ −0.22.58−0.6

]

+10

[−0.5065.6680.505

] [1.23.051.0

]

+15

[−0.5125.6930.507

] [2.43.511.4

]

results confirm the validity of the approach, issues such as the network response toa system of equations which do not have a unique solution are yet to be explored.It may be mentioned that the technique can be extended to the standard linear andquadratic programming problems.


References

1. A. Cichocki, R. Unbehauen, Neural networks for solving systems of linear equations and relatedproblems. IEEE Trans. Circuits Syst.—I 39(2), 124–138 (1992)

2. H.O. Elwan, A.M. Soliman, Novel CMOS differential voltage current conveyor and its applications.IEE Proc. Circuits Devices Syst. 144(3), 195–200 (1997)

3. J.L. Gustafson, The quest for linear equation solvers and the invention of electronic digital computing,in Proc. JVA’06, IEEE John Vincent Atanasoff 2006 International Symposium on Modern Computing(2006)

4. T.M. Hassan, S.A. Mahmoud, Fully programmable universal filter with independent gain-ωo-Q con-trol based on new digitally programmable CMOS CCII. J. Circuits Syst. Comput. 18(5), 875–897(2009)

5. D. Jiang, Analog computing for real-time solution of time-varying linear equations. Proc. ICCCAS 2,27–29 (2004)

6. S. Maheshwari, A canonical voltage-controlled VM-APS with a grounded capacitor. Circuits Syst.Signal Process. 27, 123–132 (2008)

7. J.D. Plummer, M.D. Deal, P.B. Griffin, Silicon VLSI Technology: Fundamentals, Practices and Mod-eling (Pearson Education, Upper Saddle River, 2009)

8. S.A. Rahman, Jayadeva, S.C. Duttaroy, Neural network approach to graph colouring. Electron. Lett.35(14), 1173–1175 (1999)

9. D.W. Tank, J.J. Hopfield, Simple neural optimization networks: an A/D converter, signal decisionnetwork, and linear programming circuit. IEEE Trans. Circuits Syst. CAS 33(5), 533–541 (1986)

10. J. Wang, Electronic realization of recurrent neural network for solving simultaneous linear equations.Electron. Lett. 28(5), 493–495 (1992)

11. Y. Xia, J. Wang, D.L. Hung, Recurrent neural networks for solving linear inequalities and equations.IEEE Trans. Circuits Syst.—I 46(4) (1999)

International Journal of Computer Applications (0975 – 8887)

Volume 39– No.2, February 2012

44

Non-linear Feedback Neural Network for Solution of

Quadratic Programming Problems

Mohd. Samar Ansari

Department of Electronics Engineering Aligarh Muslim University

Aligarh, India

Syed Atiqur Rahman Department of Electronics Engineering

Aligarh Muslim University Aligarh, India

ABSTRACT

This paper presents a recurrent neural circuit for solving

quadratic programming problems. The objective is

tominimize a quadratic cost function subject to

linearconstraints. The proposed circuit employs non-

linearfeedback, in the form of unipolar comparators, to

introducetranscendental terms in the energy function ensuring

fastconvergence to the solution. The proof of validity of the

energy function is also provided. The hardware complexity of

the proposed circuit comparesfavorably with other proposed

circuits for the same task. PSPICE simulation results

arepresented for a chosen optimization problem and are

foundto agree with the algebraic solution.

General Terms

Neural Networks, Quadratic Programming Problem.

Keywords

Dynamical Systems, Non-Linear Synapse, Feedback

Networks.

1. INTRODUCTION Quadratic programming problem (QPP) is the problem of

optimizing (minimizing or maximizing) a quadratic function

of several variables subject to linear constraints on these

variables. Such problems arise naturally in a variety of

applications, such as structural analysis [1], optimal control

[2], plastic analysis [3], antenna array pattern synthesis [4],

geometric optimization [5], propulsion physics [6], multi-

commodity networks [7], etc. Moreover, in the discipline of

constrained optimization, problems with nonlinear objective

functions are usually approximated by a second-order system

and solved by a standard quadratic programming technique.

Traditional methods for solving quadratic programming

problems typically involve an iterative process, but long

computational time limits their usage. An alternative approach

to solution of this problem is to exploit the artificial neural

networks (ANN's) which can be considered as an analog

computer relying on a highly simplified model of neurons [8].

ANN's have been applied to several classes of constrained

optimization problems and have shown promise for solving

such problems more effectively. For example, the Hopfield

neural network has proven to be a powerful tool for solving

some of the optimization problems. Tank and Hopfield first

proposed a neural network for solving mathematical

programming problems, where a linear programming problem

(LPP) was mapped into a closed-loop network [9]. Later, the

dynamical approach was extended for solving quadratic

programming problems. Over the past two decades several

neural-network architectures for solving quadratic

programming problems have been proposed by Kennedy &

Chua [10], Maa&Shanblatt[11], Chen & Fang [12], Wu et

al.[13] and Xia [14]. More recently, Malek&Alipour proposed

a recurrent neural network that is able to solve quadratic

programming problems [15] without needing to set network

parameters thereby reducing the number of analog multipliers

required.

In this paper, a hardware solution to the problem of solving a

quadratic programming problem is presented. The proposed

architecture uses non-linear feedback which leads to a new

energy function that involves transcendental terms. This

transcendental energy function is fundamentally different

from the standard quadratic form associated with Hopfield

network and its variants. To solve a QPP in n variables with m

constraints, the circuit requires n opamps,m unipolar

comparators and (n2+mn) resistances thereby causing the

hardware complexity of the proposed network to compare

favorably with the existing hardware implementations. It may

be mentioned that a similar approach of using non-linear

synaptic interconnections between neurons has also been

employed to solve systems of simultaneous linear equations

[16] and linear programming problems [17].

The remainder of this paper is arranged as follows. A brief

review of relevant technical literature on the solution of QPP

using neural network based methods is presented in Section-2.

Section-3 outlines the mathematical formulation of the basic

problem and details of the proposed network. Section-4

contains explanation of the energy function and the proof of

its validity. Section-5 contains the circuit implementation of

the proposed network for a set of sample problem in two

variables. PSPICE simulation results of the proposed circuit

are also presented. Issues that are expected to arise in actual

monolithic implementations are discussed in Section-6.

Concluding remarks are presented in Section-7.

2. EXISTING NEURAL NETWORKS

FOR QPP Various methods to solve QPP by employing neural network

approaches are available in the technical literature. Kennedy

& Chua extended the Tank and Hopfield network by

developing a neural network for solving nonlinear

programming problems, by satisfaction of the Karush–Kuhn–

Tucker optimality conditions [10]. However, the need to set a

penalty parameter means that the network can generate

approximate solutions only and implementation problems

arise when the penalty parameter is large. Each variable

amplifier comprises of 2 opamps, 2 resistors and 1 capacitor

whereas for satisfying each constraint, the constraint amplifier

employs 3 opamps, 2 resistors and 1 diode [10]. Wang

proposed a recurrent neural network for solving QPPs with

equality constraints. The network is asymptoticallystable and

is able to generate optimal solutions to quadratic programs

with equality constraints. An opamp based circuit realization



45

of the network is also presented which requires (n+m) neurons

for solving a QPP in n variables with m constraints. Each

neuron is made up of a summer, an integrator, and an inverter

consuming 3 opamps, 1 capacitor and (n+5) resistors [18].

Wang's network is not suitable for real time applications as it

takes around 50 ms to arrive at the solution [18]. A rigorous

analysis of the prominent neural networks for QPP, available

till that time (1992), is presented in [11]. Forti&Tesi presented

new conditions capable of ensuring existence, uniqueness, and

global asymptotic stability of the equilibrium point for

Kennedy and Chua's network [19]. Wu et al. proposed two

neural network models for solving LPP and QPP, the

convergence of which was not dependent on the network

parameters [13].

Around the same time, Xia also put forward a neural network

capable of solving both LPP and QPP in which no parameter

tuning was necessary. Moreover, the actual hardware

implementation was somewhat simplified, as compared to its

contemporaries, because of the fact that no analog multipliers

were required for the variables [14]. To solve a QPP in n

variables with m constraints, Xia's network consisted of

(2m2+4mn) amplifiers, (2m2+4mn+3m+3) summers, (n+m)

integrators, and n limiters. Tao, Cao and Sun further

simplified the network of Xia [14], and reduced the system

complexity [20]. More recently, Liu and Wang presented a

one layer feedback neural network with a discontinuous hard-

limiting activation function for solving QPP in which the

number of neurons is the same as the number of decision

variables [21]. Each neuron in [21] is composed of two

adders, (3n+1) resistors, one limiter and an integrator.

Although significant reduction in circuit complexity is

achieved, the time that the circuit takes to arrive at the correct

solution is of the order of seconds thereby making the circuit

unsuitable for applications requiring fast solution times. A

comprehensive bibliography of the technical literature related

to QPP can be found in [22].

3. PROPOSED CIRCUIT Let the second-orderfunction to be minimized be

𝐹 =

𝑉1

𝑉2

⋮𝑉𝑛

𝑇

c11 c12 … c1n

c21 c22 … c2n

⋮ ⋮ … ⋮cn1 cn2 … cnn

𝑉1

𝑉2

⋮𝑉𝑛

(1)

subject tothe following linear constraints

a11 a12 … a1n

a21 a22 … a2n

⋮ ⋮ … ⋮am1 am2 … amn

𝑉1

𝑉2

⋮𝑉𝑛

≤

b1

b2

⋮bm

(2)

whereV1, V2,…, Vn are the variables, and aij, cij and bi (i = 1, 2,

…, m; j = 1, 2, …, n) are constants. The proposed

neuralnetwork based circuit to minimize the quadratic

function given in (1) in accordance with the constraints of (2)

is presented in Fig. 1. As can be seen from Fig. 1, individual

equations from the set of equations to be solved are passed

through non-linear synapses which are realized using unipolar

comparators comprising of operational amplifiers and diodes.

Rp1 and Cp1are the input resistance and capacitance of the

opamp that is used to emulate the functionality of a neuron.

These parasitic components are included to model the

dynamic nature of the opamp. The outputs of the comparators

are fed to neurons having weighted inputs. The neurons are

realized by using opamps and the weights are implemented

using resistances. The currents arriving to the neuron from

various synapses get added up at the input of the neuron.

Fig 1: First neuron of the proposed feedback neural

network circuit to solve a quadratic programming

problem in n variables with m linear constraints.

Graphical representation of the transfer characteristics for a

bipolar comparator is shown in Fig 2(a) from where it can be

seen that the comparator output saturates at ±Vm when the two

inputs differ by more than few millivolts in magnitude.

Unipolar transfer characteristics can be obtained using an

opamp (the transfer characteristics of which can be modeled

by (3) by employing a diode as depicted in Fig. 3, the diode

essentially `trimming' one half of the transfer characteristic

curve, which are shown in Fig. 2(b) and can be

mathematically modeled by (4). As is explained in the next

section, such unipolar comparator characteristics are utilized

to obtain an energy function which acts to bring the neuronal

states to the feasible region.

Fig 2: Transfer characteristics of (a) bipolar; and

(b)unipolar; comparators

Fig 3: Obtaining unipolar comparator characteristics

using an opamp and a diode

𝑥 = 𝑉𝑚 𝑡𝑎𝑛ℎ 𝛽 𝑉𝑖 − 𝑉𝑗 (3)

𝑥 =1

2𝑉𝑚 𝑡𝑎𝑛ℎ 𝛽 𝑉𝑖 − 𝑉𝑗 + 1 (4)

Using (4), the output of the i-th unipolar comparator in Fig. 1

can be given by (5) where βis the open-loop gain of the

comparator (practically very high), ±Vm are the output voltage

levels of the comparator and V1, V2,…,Vn are the neuron

outputs.

𝑥𝑖 =𝑉𝑚

2 𝑡𝑎𝑛ℎ 𝛽 𝑎𝑖1𝑉1 + 𝑎𝑖2𝑉2 + ⋯ + 𝑎𝑖𝑛𝑉𝑛 − 𝑏𝑖 + 1 (5)



47


𝑑𝐸

𝑑𝑡= 𝐾𝐶𝑖

𝑁

𝑖=1

𝑑𝑢𝑖

𝑑𝑡

2 𝑑𝑉𝑖

𝑑𝑢𝑖

20

The transfer characteristics of the output opamp used to

implement the neurons in Fig. 1 implements the activation

function of the neuron and can be written as

𝑉𝑖 = 𝑓 𝑢𝑖 21

Fig 5: Transfer characteristics of the opamp used to

realize the neurons

whereVi denotes the output of the opamp and ui corresponds

to the internal state at the inverting terminal. The function f is

typically a saturating, monotonically decreasing one, as

shown in Fig. 5, and therefore [16],

𝑑𝑉𝑖

𝑑𝑢𝑖 ≤ 0 22

thereby resulting in

𝑑𝐸

𝑑𝑡≤ 0 23

with the equality being valid for

𝑑𝑢𝑖

𝑑𝑡= 0 24

Equation (23) shows that the energy function can never


energy function. The second criterion viz. the energy function

must have a lower bound is also satisfied for the circuit of Fig.

1 wherein it may be seen that V1, V2, …, Vn are all bounded (as

they are the outputs of opamps, as given in (21) amounting to

E, as given in (8), having a finite lower bound.

5. SIMULATION RESULTS This sectiondeals with the application of the proposed

network to task of minimizing the objective function

3𝑉12 + 4𝑉1𝑉2 + 5𝑉2

2 25

subject to

𝑉1 − 𝑉2 ≤ −1(26)

𝑉1 + 𝑉2 ≤ 1

The values of resistances acting as the weights on the neurons

are obtained from (17,18). For the purpose of simulation, the

value of K was chosen to be 1 KΩ. Using K = 1 KΩ in

(17,18) gives

Rc11 = Rc12 = Rc21 = Rc22 = K = 1 K, R11 = 1.66 K, R21 = 1 K,

R12 = 1.66 K, R22 = 1 K

For the purpose of PSPICE simulations, the unipolar voltage

comparator was realized using a diode clamp with an opamp

based comparator. The transfer characteristics obtained during

the PSPICE simulations for opamp based bipolar and unipolar

comparators are presented in Fig.6. For the purpose of this

simulation, the LMC7101A CMOS opamp model from the

Orcad library in PSPICE was utilised. The value of βfor this

opamp was measured to be 1.1×104 using PSPICE simulation.

Fig 6: Transfer characteristics for opamp based unipolar

and bipolar comparators

Routine mathematical analysis of (25) yields: V1= – 0.584,

V2= 0.416. The resultant plots of the neuron output voltages

as obtained after PSPICE simulation are presented in Fig.7

from where it can be seen that V(1) = –0.58 V and V(2) =

0.41 V which are very near to the algebraic solution thereby

confirming the validity of the approach. The initial node

voltages were kept as V(1) = –1 mV and V(2) = –10 mV.

Fig 7: Simulation results for the proposed circuit applied

to minimize (25) subject to (26)

6. ISSUES IN VLSI IMPLEMENTATION This section deals with the monolithic implementation issues

of the proposed circuit. The PSPICE simulations assumed that

all operational amplifiers (and diodes) are identical, and

therefore, it is required to determine how deviations from this

assumption affect the performance of the network. Effects of

variations in component values from one neuron to another

were also investigated using Monte-Carlo analysis in PSPICE.

A 10% tolerance with Gaussian deviation profile was put on

the resistances used in the circuit to solve (25). The analysis

was carried out for 100 runs and the Mean Deviation was

found out to be -144.73×10-6 and Mean Sigma (Standard

Deviation) was 0.0119. Offset analysis was also carried out by

incorporating random offset voltages (in the range of 1 mV to

10 mV) to the opamps. The Mean Deviation in this case was

measured to be -143.51×10-6and the Mean Sigma (Standard

Deviation) was 0.012. As can be seen, the effects of

mismatches and offsets on the overall precision of the final

results are in an acceptable range.

In fact, the realization of unipolar comparators by the use of

opamps in the proposed circuit tends to increase the circuit

complexity. The transistor count can be further reduced by

utilising voltage-mode unipolar comparators instead of the

opamp-diode combination. This also suggests that a real, large

scale implementation for solving quadratic programming

problems with high variable counts might be quite different.



48

Alternative realizations based on the differential equations (6)

governing the system of neurons are being investigated. Other

approaches to obtain the tanh(.) non-linearity include the use

of a MOSFET operated in the sub-threshold region [24] and

the use of Current Differencing Transconductance Amplifier

(CDTA) to provide the same nonlinearity in the current-mode

regime [25].

7. CONCLUSION In this paper, a CMOS compatible approach to solve a

quadratic programming problem in n variables subject to

mlinear constraints, which uses nneurons and msynapses is

presented. Each neuron requires one opamp and each synapse

is implemented using a unipolar voltage-mode comparator.

This results in significant reduction in hardware over the

existing schemes. The proposed network was tested on a

sample problem of minimizing a quadratic function in 2

variables and the simulation results confirm the validity of the

approach.

8. REFERENCES [1] Atkociunas, J. 1996. Quadratic programming for

degenerate shakedown problems of bar structures. Mechanics Research Communications, 23(2), 195–206.

[2] Bartlett, R.A., Wachter, A., and Biegler, L.T. 2000. Active set vs. interior point strategies for model predictive control. In Proceedings of the American Control Conference, Chicago, USA, June 2000, 4229–4233.

[3] Maier, G. and Munro, J. 1982. Mathematical programming applications to engineering plastic analysis. Applied Mechanics Reviews, 35, 1631–1643.

[4] Nordebo, S., Zang, Z., and Claesson, I. 2001. A semi-infinite quadratic programming algorithm with applications to array pattern synthesis. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 48(3), 225–232.

[5] Schonherr, S. 2002. Quadratic programming in geometric optimization: theory, implementation, and applications. Technical report, Swiss Federal Institute of Technology, Zurich.

[6] Borguet, S. and O. Lonard, O. 2009. A quadratic programming framework for constrained and robust jet engine health monitoring. Progress in Propulsion Physics, 1, 669–692.

[7] Dembo, R.S. and Tulowitzki, U. 1988. Computing equilibria on large multicommodity networks: An application of truncated quadratic programming algorithms. Networks, 18(4), 273–284.

[8] Krogh, A. 2008. What are artificial neural networks? Nature Biotechnology, 26(2), 195–197.

[9] Tank, D.W. and Hopfield, J. 1986. Simple ‘neural’ optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit. IEEE Transactions on Circuits and Systems, 33(5), 533–541.

[10] Kennedy, M.P. and Chua, L.O. 1988. Neural networks for nonlinear programming. IEEE Transactions on Circuits and Systems, 35(5), 554–562.

[11] Maa, C.-Y. andShanblatt, M.A. 1992. Linear and quadratic programming neural network analysis. IEEE Transactions on Neural Networks, 3(4), 580–594.

[12] Chen, Y.-H. and Fang, S.-C. 1998. Solving convex programming problems with equality constraints by neural networks. Computers & Mathematics with Applications, 36(7), 41–68.

[13] Wu, X.-Y., Xia, Y.-S., Li, J., and Chen, W.-K. 1996. A high-performance neural network for solving linear and quadratic programming problems. IEEE Transactions on Neural Networks, 7(3), 643–651.

[14] Xia Y. 1996. A new neural network for solving linear and quadratic programming problems. IEEE Transactions on Neural Networks, 7(6), 1544–1548.

[15] Malek, A. and Alipour, M. 2007. Numerical solution for linear and quadratic programming problems using a recurrent neural network. Applied Mathematics and Computation, 192(1), 27–39.

[16] Rahman, S.A. and Ansari, M.S. 2011. A neural circuitwith transcendental energy function for solving system of linear equations. Analog Integrated Circuits and Signal Processing, 66, 433–440.

[17] Ansari, M.S. and Rahman, S.A. 2010. A DVCC-based non-linear analog circuit for solving linear programming problems. In Proceedings of International Conference on Power, Control and Embedded Systems (ICPCES), Dec 2010, 1–4.

[18] Wang, J. 1992. Recurrent neural network for solving quadratic programming problems with equality constraints. Electronics Letters, 28(14), 1345–1347.

[19] Forti, M. and Tesi, A. New conditions for global stability of neural networks with application to linear and quadratic programming problems. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 42(7), 354–366.

[20] Tao, Q., Cao, J., and Sun, D. 2001. A simple and high performance neural network for quadratic programming problems. Applied Mathematics and Computation, 124(2), 251–260.

[21] Liu, Q. and Wang, J. 2008. A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming. IEEE Transactions on Neural Networks, 19(4), 558–570.

[22] Gould, N.I.M., and Toint, P.L. 2010. A quadratic programming bibliography. Technical report, RAL Numerical Analysis Group, March 2010.

[23] Rahman, S.A., Jayadeva, and S.C. Dutta Roy. 1999. Neural network approach to graph colouring. Electronics Letters, 35(14), 1173–1175.

[24] Newcomb, R.W. and Lohn, J.D. 1998. The handbook of brain theory and neural networks. Chapter: Analog VLSI for neural networks, MIT Press, Cambridge, MA, USA, 86–90.

[25] Ansari, M.S. and Rahman, S.A. 2009. A novel current-mode non-linear feedback neural circuit for solving linear equations. In Proceedings of International Conference on Multimedia, Signal Processing and Communication Technologies, 284–287.



31

A Non–linear Feedback Neural Network for

Graph Coloring

Mohd. Samar Ansari Department of Electronics

Engineering Aligarh Muslim University

Aligarh, India

Syed Atiqur Rahman Department of Electronics


Aligarh, India

Syed Javed Arif Department of Electronics


Aligarh, India

ABSTRACT

A feedback neural network for solving graph coloring

problem is presented. The circuit has an associated

transcendental energy function that ensures fast convergence

to the exact solution. Hardware and PSPICE simulation

results on random and benchmark problems have been

presented. Test results are compared with existing techniques

for graph coloring to show that the proposed neural network

model provides a significant reduction in the number of colors

while enjoying a simple and efficient circuit implementation.

General Terms

Neural Networks, Graph Coloring.

Keywords

Neural network applications, Neural network hardware, Non-

linear circuits, Graph theory,Dynamical Systems, Non-Linear

Synapse,Feedback Networks.

1. INTRODUCTION The graph coloring problem (GCP) has a large number of

applications such as register allocation in digital computers,

frequency channel assignment in mobile communication,

and layer assignment in VLSI design [1-3]. In GCP,

values or colors are assigned to different nodes of a

graph such that no two adjacent nodes get the same

color, and the number of colors are also minimized. The

Hopfield Neural Network (HNN) and its

modifications have been applied to solve GCP, but the

implementation requires N2 neurons and at most N4

weights for coloring a graph of N nodes [4]. This puts

severe constraints on the feasibility of HNN realization in

hardware. Another feedback neural network architecture

which uses non-linear synapses, and has a transcendental

Energy function, requires N neurons and at most N2

interconnections for solving a problem with N nodes [4]. In

this paper, a neural circuit is presented which modifies the

Energy function of [4] to eliminate those local minima that

correspond to solutions with large number of colors. It is seen

that the number of colors are reduced thereby providing a

significant improvement in solution quality. Moreover, the

circuit is suitable for VLSI implementation as compared to an

existing scheme which employs one coupled relaxation

oscillator for each node in the graph to be colored [5].

2. PROPOSED CIRCUIT FOR GRAPH

COLORING Fig. 1 shows thei-th neuron of proposed network. In the

proposed circuit, output voltages of different neurons

represent the colors of different nodes. Ci and ri denote the

internal capacitance and resistance of the i-th neuron

respectively, ui is the internal state and Rii is the self–feedback

resistance of i-th neuron. The output of other neurons Vj (j =

1, 2…N) are connected to the input of i-th neuron through

unipolar comparators.

ri Ci

Rii

x i1

xij

xiN

Vi

V1

Vj

VN

Ri1

Ri j

RiN

ui

Vi

Ni

Fig 1: i-th neuron of the proposed neural network for

graph coloring

The self – feedback resistance (Rii) is given by

𝑅𝑖𝑖 = 𝑅𝑐

∆ (1)

whereRc is a constant resistance and ∆ is the maximum degree

of the graph. Next, we define a constant aij such that

𝑎𝑖𝑗 = 1; 𝑖 − 𝑡ℎ 𝑛𝑜𝑑𝑒 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑒𝑑 𝑡𝑜 𝑗 − 𝑡ℎ 𝑛𝑜𝑑𝑒0; 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

(2)

Using (2), the feedback resistance of the i-th neuron is given

by

𝑅𝑖𝑗 = 𝑅𝑐

𝑎𝑖𝑗 (3)

From fig. 1, xij can be written as

𝑥𝑖𝑗 =𝑉𝑚

2 𝑡𝑎𝑛ℎ 𝛽 𝑉𝑗 − 𝑉𝑖 − 1 (4)

where β is the open-loop gain of the comparator (practically

very high) and ±Vm are the voltage levels of the comparator

output.

Node equation for node „Ni‟ gives the equation of motion of

the i-th neuron in the state space as

𝐶𝑖𝑑𝑢 𝑖

𝑑𝑡=

𝑥𝑖𝑗

𝑅𝑖𝑗

𝑁𝑗=1𝑗≠𝑖

+ 𝑉𝑖

𝑅𝑖𝑖 −

𝑢 𝑖

𝑅𝑖(5)



32

where

1

𝑅𝑖=

1

𝑅𝑖𝑗

𝑁𝑗=1𝑗≠𝑖

+ 1

𝑅𝑖𝑖+

1

𝑟𝑖(6)

Moreover, it can be shown that the Energy Function „E‟ of the

network in Fig. 1 is

𝐸 = 1

2

𝑉𝑖2

2𝑅𝑖𝑖

𝑁𝑖=1 −

𝑉𝑚

4𝛽𝑅𝑐

𝑎𝑖𝑗𝑁𝑗=1𝑗≠𝑖

𝑁𝑖=1 ln cosh 𝛽 𝑉𝑗 − 𝑉𝑖 −

𝑉𝑚

2𝑅𝑐

𝑎𝑖𝑗𝑁𝑗=1𝑗≠𝑖

𝑁𝑖=1 𝑉𝑖 −

1

𝑅𝑖 𝑢𝑖𝑉𝑖

0𝑑𝑉𝑁

𝑖=1 (7)

Fig 2: Plot of the second and third terms in equation (7)

The last term in (7) is usually neglected for high values of the

open-loop gain of the opamp used to realize the neurons. The

first term on the right hand side of (7) is quadratic which tries

to minimize the number of colors. The second term has got a

negative sign. Therefore, the energy function (E) will be

minimized if second term is maximized. This happens when

the voltages corresponding to connected nodes in a graph are

far away from each other. The first two terms on the right

hand side are balancing each other to color a graph properly.

The third term also contributes to lowering of number of

different colors by eliminating all those local minima in the

energy function for which node voltages are negative. The

combined effect of the second and third terms in (7) for a

typical 2-node problem is shown in Fig. 2.

3. SIMULATION RESULTS The proposed network was tested for various random graphs

as well as a standard benchmark problem for graph coloring,

myciel3.col [6]. Breadboard implementation as well as

PSPICE simulation was performed. The results of the tests are

given in table 1 from which it is seen that the proposed

network gives a solution to all the problems tested and in all

the cases the solution is very near to the chromatic number of

the graph. Simulation runs of the proposed network for the

myciel3.col benchmarking problem returned the best solution

of 5 colors which is very near to the chromatic number for

myciel3.col, that being 4. The performance of the proposed

network was also compared with a network proposed earlier

[4]. Table 2 gives this performance comparison. It is evident

that substantial improvements have been achieved both in the

best and the average solutions for most examples.

Table1.Hardware and PSPICE simulation test results for the proposed network



33

4. DISCUSSION This section deals with the monolithic implementation issues

of the proposed circuit. The PSPICE simulations assumed that

all operational amplifiers are identical, and therefore, analysis

is required to determine how deviations from this assumption

affect the performance of the network. While the component

values used for the capacitors and resistors affect the

convergence time, in a real implementation, they would also

need to be chosen keeping in mind limitations of the active

devices to source or sink currents. Effects of variations in

component values from one neuron to another also need to be

investigated. In fact, the drawbacks of the use of opamps in

the proposed circuit suggest that a real, large scale

implementation of the circuit for colouring a graph with large

number of nodes might be quite different. Alternative

realizations based on the differential equations (5) governing

the system of neurons are being investigated. An

implementation based on operational transconductance

amplifiers (OTAs) would appear to be one viable alternative;

similarly, circuits using MOS transistors operating in the

subhreshold regime would be another. In the latter case, the

tanh(.) nonlinearity is easily obtained because of the

transistor‟s current–voltage characteristic [7] and is expected

to be the focus of future work. However, the present

manuscript focuses mainly on the principle of such a network,

and a VLSI implementation is beyond the scope of this paper.

5. CONCLUSION A feedback neural network based architecture to solve graph

coloring problem is presented. Test results of hardware

implementation and PSPICE simulation for five different

graphs including the myciel3.col benchmark problem are

presented. Further, PSPICE simulations of the proposed

circuit show improvements in both the minimum and the

average number of colors over its predecessor.

6. REFERENCES [1] Zeitlhofer, T. and Wess, B. 2004. A Comparison of

Graph Coloring Heuristics for Register Allocation based

on Coalescing in Interval Graphs. Proc. ISCAS 04, vol.

IV, 529 – 532.

[2] El-Fishawy, N.A., Hadhood, M. M., Elnoubi, S., and El-

Sersy, W.2000. A modified Hopfield neural network

algorithm for cellular radio channel assignment. Proc.

TENCON 2000, 2, 213 – 216.

[3] Blas, A.D., Jagota, A., and Hughey, R. 2002. Energy

Function – Based Approaches to Graph Coloring. IEEE

Trans. On Neural Networks, 13, 81–91.

[4] Rahman, S.A., Jayadeva and Dutta Roy, S.C. 1999.

Neural network approach to graph colouring. Electronics

Letters, 35(14), 1173–1175.

[5] Wu, C.W. 1998. Graph Coloring via Synchronization of

Coupled Oscillators. IEEE Trans. Circuits Syst. I, 45(9),

974–978.

[6] Lui, J., Zhong, W., and Jiao, L.2006. Comments on “The

1993 DIMACS Graph Coloring Challenge” and “Energy

Function-Based Approaches to Graph Coloring” IEEE

Trans. On Neural Networks, 17(2), 533.

[7] Newcomb, R.W. and Lohn, J.D.1998. Analog VLSI for neural networks in The Handbook of Brain Theory and Neural Networks. Cambridge, MA: The MIT Press.

Table2.Performance comparison of the proposed and existing networks

A Non – Linear Feedback Neural Network for Solving Quadratic

Programming Problems

MOHD. SAMAR ANSARI

1,* AND S.A. RAHMAN2

1,2

Department of Electronics Engineering, Aligarh Muslim University, Aligarh, India

Emails: [email protected], [email protected]

2

Abstract- This paper presents a neural circuit for solving quadratic programming problems. The objective is to minimize

a quadratic cost function subject to linear constraints. The proposed circuit employs non-linear feedback, in the form of

unipolar comparators, to introduce transcendental terms in the energy function ensuring fast convergence to the

solution. PSPICE simulation results are presented for a chosen optimization problem and are found to agree with the

algebraic solution.

Keywords: Neural Networks, Non – Linear circuits, Feedback Neural Networks, Quadratic Programming, Dynamical

Systems

1. Introduction

In the discipline of constrained optimization, problems with nonlinear objective functions are usually

approximated by a second-order (quadratic) system and solved approximately by a standard quadratic

programming technique. Traditional methods for solving quadratic programming problems typically involve

an iterative process, but long computational time limits their usage. There is an alternative approach to

solution of this problem. It is to exploit the artificial neural networks (ANN's) which can be considered as an

analog computer relying on a highly simplified model of neurons [1]. ANN's have been applied to several

classes of constrained optimization problems and have shown promise for solving such problems more

effectively. For example, the Hopfield neural network has proven to be a powerful tool for solving some of

the optimization problems. Tank and Hopfield first proposed a neural network for solving

mathematical programming problems, where a linear programming problem was mapped into a closed-

loop network [2]. Later, the dynamical approach was extended for solving quadratic programming

problems. Over the past two decades several neural-network architectures for solving quadratic

programming problems have been proposed by Kennedy and Chua [3], Maa and Shanblatt [4], Chen and

Feng [5], Wu et al. [6], Xia [7]. These networks depend on the network parameters or use expensive analog

multipliers. More recently, Malek and Alipour [8] proposed a recurrent neural network that is able to solve

quadratic programming problems. This architecture is advantageous in the sense that there is no need to set

network parameters. Moreover, the number of analog multipliers is reduced. This reduces the hardware

complexity to a certain extent. However, the network still has a complex hardware in the sense that the

main processing element (alpha) would be cumbersome to implement in hardware. In this paper, we present

a hardware solution to the problem of solving a quadratic programming problem subject to linear constraints.

The proposed architecture uses non-linear feedback which leads to a new energy function that involves

transcendental terms. This transcendental energy function is fundamentally different from the standard

quadratic form associated with Hopfield network and its variants. The hardware complexity of the proposed

circuit compares favourably with the existing hardware implementations.

The remainder of this paper is arranged as follows: Section – 2 outlines the basic problem and the

mathematical formulation on which the development of the proposed network will be based. Section – 3

contains the circuit implementation of the proposed network for a set of sample problem in two variables.

SPICE simulation results of the proposed circuit are also presented. Issues that are expected to arise in actual

monolithic implementations are discussed in Section – 4. Concluding remarks are presented in Section – 5.

2. Proposed Circuit

Let the function to be minimized be

𝐹 = 𝐶𝑖𝑗𝑉𝑖𝑉𝑗𝑛𝑗=1

𝑛𝑖=1 (1)

Subject to

a11 a12 … a1n

a21 a22 … a2n

⋮ ⋮ … ⋮am1 am2 … amn

𝑉1

𝑉2

⋮𝑉𝑛

≤

b1

b2

⋮bm

(2)

Where V1, V2,..., Vn are the variables and aij, cij and bi are constants. The proposed neural-network based

circuit to minimize the quadratic function given in (1) in accordance with the constraints of (2) is presented in

fig 1. As can be seen from fig. 1, individual equations from the set of equations to be solved are passed

through non-linear synapses which are realized using comparators. The comparator outputs can be modelled

by equations (3) given below.

𝑥1 =𝑉𝑚

2 𝑡𝑎𝑛ℎ 𝛽 𝑎11𝑉1 + 𝑎12𝑉2 + ⋯ + 𝑎1𝑛𝑉𝑛 − 𝑏1 + 1

𝑥2 =𝑉𝑚

2 𝑡𝑎𝑛ℎ 𝛽 𝑎21𝑉1 + 𝑎22𝑉2 +⋯ + 𝑎2𝑛𝑉𝑛 − 𝑏2 + 1 (3)

⋮

𝑥𝑚 =𝑉𝑚

2 tanh 𝛽 𝑎𝑚1𝑉1 + 𝑎𝑚2𝑉2 +⋯ + 𝑎𝑚𝑛 𝑉𝑛 − 𝑏𝑚 + 1

where 𝛽 is the open-loop gain of the comparator (practically very high), ±Vm are the output voltage levels of

the comparator and V1, V2,..., Vn are the neuron outputs. The outputs of the comparators are fed to neurons

having weighted inputs. The neurons are realized by using opamps and the weights are implemented using

transconductance amplifiers. The currents arriving to the neuron from various synapses get added up at the

input of the neuron. Rpi and Cpi are the input resistance and capacitance of the opamp corresponding to the

i-th neuron. These parasitic components are included to model the dynamic nature of the opamp.

Vi

Vout

+

_

Vi

Vout

+

_

+

_

Vi

b1

b2

x1

x2

Rpi Ci

Vi

Vout

+

_bnxm

G1

G2

Gn

a11V1+ a12V2 + ...+a1nVn

a21V1+ a22V2 +...+ a2nVn

am1V1+ am2V2 + ...+amnVn

g1i x1

g2i x2

gmi xm

To other

neurons

V1 V2 Vn

R1i R2i Rni

Ni

Fig. 1 i-th neuron of the proposed feedback neural network circuit to solve a quadratic programming problem in n-

variables with m- linear constraints

Applying node equations for node ‘Ni’ we can write the equations of motion of the various neurons. For the

first neuron,

𝐶1

𝑑𝑢1

𝑑𝑡= 𝑥1𝑔𝑐11 + 𝑥2𝑔𝑐21 + ⋯ + 𝑥𝑚𝑔𝑐𝑚1 +

𝑉1

𝑅11+

𝑉2

𝑅21+ ⋯ +

𝑉𝑛𝑅𝑛1

− 𝑢1 𝑔𝑐11 + 𝑔𝑐21 + ⋯ + 𝑔𝑐𝑚1 +1

𝑅11+

1

𝑅21+ ⋯+

1

𝑅𝑛1+

1

𝑅𝑝1 (4)

Simplifying, we get

𝐶1

𝑑𝑢1

𝑑𝑡= 𝑥1𝑔𝑐11 + 𝑥2𝑔𝑐21 + ⋯ + 𝑥𝑛𝑔𝑐𝑚1 +

𝑉1

𝑅11+

𝑉2

𝑅21+ ⋯ +

𝑉𝑛𝑅𝑛1

− 𝑢1 1

𝑅1 (5)

Where

1

𝑅1=

1

𝑅11+

1

𝑅21+ ⋯ +

1

𝑅𝑛1+ 𝑔𝑐11 + 𝑔𝑐21 + ⋯ + 𝑔𝑐𝑚1 +

1

𝑅𝑝1 (6)

Similarly, we have for the second neuron

𝐶2

𝑑𝑢2

𝑑𝑡= 𝑥1𝑔𝑐12 + 𝑥2𝑔𝑐22 + ⋯+ 𝑥𝑛𝑔𝑐𝑚2 +

𝑉1

𝑅12+

𝑉2

𝑅22+ ⋯ +

𝑉𝑛𝑅𝑛2

− 𝑢2 1

𝑅2 (7)

Where

1

𝑅2=

1

𝑅12+

1

𝑅22+ ⋯ +

1

𝑅𝑛2+ 𝑔𝑐12 + 𝑔𝑐22 + ⋯ + 𝑔𝑐𝑚2 +

1

𝑅𝑝2 (8)

And, for the n-th neuron we get,

𝐶𝑛

𝑑𝑢𝑛

𝑑𝑡= 𝑥1𝑔𝑐1𝑛 + 𝑥2𝑔𝑐2𝑛 + ⋯+ 𝑥𝑛𝑔𝑐𝑚𝑛 +

𝑉1

𝑅1𝑛+

𝑉2

𝑅2𝑛+ ⋯ +

𝑉𝑛𝑅𝑛𝑛

− 𝑢𝑛 1

𝑅𝑛 (9)

Where

1

𝑅𝑛=

1

𝑅1𝑛+

1

𝑅2𝑛+ ⋯ +

1

𝑅𝑛𝑛+ 𝑔𝑐1𝑛 + 𝑔𝑐2𝑛 + ⋯ + 𝑔𝑐𝑚𝑛 +

1

𝑅𝑝𝑛 (10)

Where ui is the internal state of the i-th neuron and gcji is the voltage-to-current conversion factor of the j-th

transconductance block for the i-th output current. As is shown later in this section, these weights are

governed by the entries in the coefficient matrix of (2).

Moreover, the network in fig. 1 can be associated with an Energy Function ‘E’ of the form

𝐸 = 𝐶𝑖𝑗𝑉𝑖𝑉𝑗

𝑛

𝑗=1

𝑛

𝑖=1

+ 𝑉𝑚2

𝑎𝑖𝑗𝑉𝑗

𝑛

𝑗=1

𝑚

𝑖=1

+ 𝑉𝑚2𝛽

ln coshβ (𝑎𝑖𝑗𝑉𝑗

𝑛

𝑗=1

− 𝑏𝑖)

𝑚

𝑖=1

(11)

This expression of the Energy Function can be written in a slightly different (but more illuminating) form as

𝐸 = 𝐶𝑗𝑖𝑉𝑖𝑉𝑗

𝑛

𝑗=1

𝑛

𝑖=1

+ 𝑃1 + 𝑃2 + ⋯ + 𝑃𝑚 (12)

Where P1, P2, ..., Pm are the penalty terms. The penalty terms can separately be given as

𝑃1 = 𝑉𝑚2

𝑎11𝑉1 + 𝑎12𝑉2 + ⋯ + 𝑎1𝑛𝑉𝑛 +𝑉𝑚2𝛽

ln coshβ( 𝑎11𝑉1 + 𝑎12𝑉2 + ⋯ + 𝑎1𝑛𝑉𝑛 − 𝑏1) (13)

Similarly, the remaining penalty functions can be written, as given below.

𝑃2 = 𝑉𝑚2

𝑎21𝑉1 + 𝑎22𝑉2 + ⋯ + 𝑎2𝑛𝑉𝑛 +𝑉𝑚2𝛽

ln coshβ(𝑎21𝑉1 + 𝑎22𝑉2 + ⋯ + 𝑎2𝑛𝑉𝑛 − 𝑏2) (14)

And finally,

𝑃𝑚 = 𝑉𝑚2

𝑎𝑚1𝑉1 + 𝑎𝑚2𝑉2 + ⋯ + 𝑎𝑚𝑛 𝑉𝑛 +𝑉𝑚2𝛽

ln coshβ( 𝑎𝑚1𝑉1 + 𝑎𝑚2𝑉2 + ⋯+ 𝑎𝑚𝑛 𝑉𝑛 − 𝑏𝑚 ) (15)

Obtaining a partial differentiation of the combined penalty term, P w. r. t. to V1 we get

𝜕𝑃

𝜕𝑉1=

𝜕𝑃1

𝜕𝑉1+

𝜕𝑃2

𝜕𝑉1+ … +

𝜕𝑃𝑚𝜕𝑉1

(16)

𝜕𝑃

𝜕𝑉1=

𝑉𝑚2

𝑎11 + 𝑎21 + ⋯ + 𝑎𝑚1

+ 𝑎11 tanh β( 𝑎11𝑉1 + 𝑎12𝑉2 + ⋯ + 𝑎1𝑛𝑉𝑛 − 𝑏1)

+ 𝑎21 tanh β( 𝑎21𝑉1 + 𝑎22𝑉2 + ⋯ + 𝑎2𝑛𝑉𝑛 − 𝑏2) + ⋯ +𝑎𝑚1 tanh β( 𝑎𝑚1𝑉1 + 𝑎𝑚2𝑉2 + ⋯

+ 𝑎𝑚𝑛 𝑉𝑛 − 𝑏𝑚 ) (17)

Which can be simplified to,

𝜕𝑃

𝜕𝑉1= 𝑎11𝑥1 + 𝑎21𝑥2 + ⋯ + 𝑎𝑚1𝑥𝑚 (18)

Using the above relations to find the derivative of the Energy Function w. r. t. to V1 we have

𝜕𝐸

𝜕𝑉1=

𝜕

𝜕𝑉1 𝐶𝑗𝑖𝑉𝑖𝑉𝑗

𝑛

𝑗=1

𝑛

𝑖=1

+ 𝜕𝑃

𝜕𝑉1 (19)

Which yields

𝜕𝐸

𝜕𝑉1= 𝑐11𝑉1 + 𝑐12𝑉2 + ⋯ + 𝑐1𝑛𝑉𝑛 + 𝑎11𝑥1 + 𝑎21𝑥2 + ⋯ + 𝑎𝑚1𝑥𝑚 (20)

Also, if ‘E’ is the Energy Function, it must satisfy the following condition *9].

𝜕𝐸

𝜕𝑉𝑖= 𝐾𝐶𝑖

𝑑𝑢𝑖

𝑑𝑡 (21)

Where ‘K’ is a constant of proportionality and has the dimensions of resistance. Equation (x) applied to the

first neuron results in

𝑔𝑐11 =𝑎11

𝐾

𝑔𝑐21 =𝑎21

𝐾 (22)

⋮

𝑔𝑐𝑚1 =𝑎𝑚1

𝐾

A similar comparison of the remaining partial fractions for the remaining variables yields the following:

𝑔𝑐11 𝑔𝑐12

𝑔𝑐21 𝑔𝑐22

… 𝑔𝑐1𝑛

… 𝑔𝑐2𝑛

⋮ ⋮𝑔𝑐𝑚1 𝑔𝑐𝑚2

… ⋮… 𝑔𝑐𝑚𝑛

=1

𝐾

𝑎11 𝑎12

𝑎21 𝑎22

… 𝑎1𝑛

… 𝑎2𝑛

⋮ ⋮𝑎𝑚1 𝑎𝑚2

… ⋮… 𝑎𝑚𝑛

(23)

And,

𝑅11 𝑅12

𝑅21 𝑅22

… 𝑅1𝑛

… 𝑅2𝑛

⋮ ⋮𝑅𝑛1 𝑅𝑛2

… ⋮… 𝑅𝑛𝑛

= 𝐾

1

𝑐11 1

𝑐12

1𝑐21

1𝑐22

… 1𝑐1𝑛

… 1𝑐2𝑛

⋮ ⋮1

𝑐𝑛1 1

𝑐𝑛2

… ⋮

… 1𝑐𝑛𝑛

(24)

3. Simulation Results

This section deals with the application of the proposed network to task of minimizing the objective function

3𝑉12 + 4𝑉1𝑉2 + 5𝑉2

2 (25) subject to

𝑉1 − 𝑉2 ≤ −1 (26)

𝑉1 + 𝑉2 ≤ 1 The values of resistances acting as the weights on the neurons are obtained from (24, 25). For the purpose of

simulation, the value of ‘K’ was chosen to be 1 KΩ. Using K = 1 KΩ in (23, 24) gives

gc11 = gc12 = gc21 = gc22 = 1/K = 1

R11 = 1.66 KΩ, R21 = 1 KΩ, R12 = 1.66 KΩ, R22 = 1 KΩ

For the purpose of HSPICE simulations, the voltage comparator and the multi-output transconductance

amplifier for each neuron were realized as a single block using a slightly modified circuit of the differential-

input, high-gain, active-loaded transconductance amplifier [10] shown in fig. 2. Standard BSIM3 0.35µm

parameters were used for the purpose of simulations. The supply voltages were set to VDD = 15V, VSS = 0V

and VBB = 1V. The aspect ratios of the NMOS and PMOS transistors were taken to be 1.4µm/1.4µm and

2.8µm/1.4µm respectively. Further, to get the output currents according to (23) the W/L ratios of transistors

M6, M7, M8 and M9 were set to provide the required current scaling. Choosing K=1 and using (23) results in

Iout1 = Iout2 for G1 block and Iout1 = Iout2 for the G2 block.

M2

M1

M3

M4

M5

M6

M7

VDD

VSS

VBB

V2

V1 I

out1

M8

M9

Iout2

Fig. 2 CMOS implementation of the comparator and transconductance blocks

Fig. 3 Simulation results for the proposed circuit applied to minimize (25) subject to (26)

Routine mathematical analysis of (25) yields: 𝑉1= -0.584, 𝑉2= 0.416. The results of PSPICE simulation are

presented in fig. 3. From the plots of the neuron output voltages, it can be seen that V(1) = -0.58 V and V(2) =

0.41 V which are very near to the algebraic solution thereby confirming the validity of the approach.

4. Issues in actual implementation

This section deals with the monolithic implementation issues of the proposed circuit. The PSPICE simulations

assumed that all operational amplifiers (and transconductance amplifiers) are identical, and therefore,

analysis is required to determine how deviations from this assumption affect the performance of the

network. Effects of variations in component values from one neuron to another also need to be investigated.

Moreover, for obtaining weighted (scaled) current outputs use was made of the (W/L) ratios of the

transistors M6, M7, M8 and M9 of fig. 2. This results in a particular circuit implementation ‘hard-wired’ to

solve a given quadratic programming problem. Methods to scale the currents by a mechanism external to

the circuit need to be explored to impart flexibility to the circuit. Alternative realizations based on the

differential equations (9) governing the system of neurons are also being considered. Other approaches to

obtain the tanh(.) nonlinearity include the use of a MOSFET operated in the subthreshold region [10] and the

use of Current Differencing Transconductance Amplifier (CDTA) to provide the same nonlinearity in the

current-mode regime [11].

5. Conclusion

In this paper we have described a CMOS compatible approach to solve a quadratic programming problem in

n variables subject to m linear constraints, which uses n-neurons and m-synapses. Each neuron requires one

opamp and each synapse is implemented using a differential transconductance element. This results in

significant reduction in hardware over the existing schemes [3-8]. The proposed network was tested on a

sample problem of minimizing a quadratic function in 2 variables and the simulation results confirm the

validity of the approach.

REFERENCES

[1] J.J. Hopfield, D.W. Tank, (1985) ‘Neural’ computation of decisions optimization problems, Biological

Cybern., 52, 141-152,.

[2] D.W. Tank, J.J. Hopfield, (1986) Simple neural optimization networks: an A/D converter, signal decision

network, and linear programming circuit, IEEE Trans. Circ. Syst. CAS, 33, 533–541.

[3] M.P. Kennedy, L.O. Chua, (1988) Neural networks for nonlinear programming, IEEE Trans. Circ. Syst.,

35, 554–562.

[4] C.Y. Maa, M.A. Shanblatt, (1992) A two-phase optimization neural network, IEEE Trans. Neural

Network 3 (6), 580 – 594.

[5] Y.-H. Chen, S.-C. Fang, (1998) Solving convex programming problems with equality constraints by

neural networks, Comput. Math. Appl., 36 (7), 41–68.

[6] X.Y. Wu, Y.S. Xia, J. Li, W.K. Chen, (1996) A high performance neural network for solving linear and

quadratic programming problems, IEEE Trans. Neural Networks, 7 (3), 643–651.

[7] Y. Xia, (1996) A new neural network for solving linear and quadratic programming problems, IEEE

Trans. Neural Networks 7 (6), 1544–1547.

[8] A. Malek, M. Alipour, (2007), Numerical solution for linear and quadratic programming problems using a

recurrent nueral network, Appl. Math. Computation, 192, 27 – 39.

[9] S. A. Rahman, Jayadeva, S. C. Dutta Roy, (1999) Neural network approach to graph colouring,

Electronics Letters, 35, 1173 – 1175.

[10] R. W. Newcomb, J. D. Lohn, (1998) Analog VLSI for neural networks, The Handbook of Brain Theory

and Neural Networks. Cambridge, MA: The MIT Press.

[11] M. S. Ansari, S. A. Rahman. (2009) A Novel Current-mode Non-Linear Feedback Neural Circuit for

Solving Linear Equations, Proc. IMPACT – 2009, Aligarh, India, 284 – 287.

In: CMOS Technology ISBN: 978-1-61761-325-8

Editor: Min-jun Kwon © 2010 Nova Science Publishers, Inc.

Chapter 3

LINEAR AND NON-LINEAR APPLICATIONS

OF CMOS DVCC

Sudhanshu Maheshwari, Mohd. Samar Ansari

and Syed Atiqur Rahman Department of Electronics Engineering,

Aligarh Muslim University, Aligarh, India

ABSTRACT

The differential voltage current conveyor (DVCC) has recently emerged as a

versatile building block for the realization of a variety of analog signal processing

functions including, but not limited to, analog filters, amplifiers, integrators, etc. Most of

these applications can be classified as linear although non-linear applications like the

voltage-mode comparator, function generators and oscillators are also reported in the

technical literature.

This chapter presents several new and traditional applications of DVCC including

both linear and non-linear ones. Amongst the linear applications, voltage-mode and

current-mode analog filters, amplifiers and integrators are reviewed. Similarly, amongst

the available non-linear ones, oscillators and negative resistance converter are reviewed

whereas DVCC-based digital logic gates, linear equation solver and circuits for solving

linear & quadratic programming problems are proposed. Real device simulations on some

of the newly presented circuits are also given. The material presented is intended to

explore full potential of DVCC covering a broad range of practical circuit applications.

I. INTRODUCTION

Over the past few decades, a variety of active building blocks (including, but not limited

to, Current Conveyors, Current-Controlled Conveyors, Operational Transconductance

Amplifiers, Current Differencing Transconductance Amplifier, Dual-X Current Conveyor,

etc.) and their applications have been added to the technical literature [1-5]. Amongst these,

Sudhanshu Maheshwari, Mohd. Samar Ansari and Syed Atiqur Rahman

2

the Second Generation Current Conveyor (CCII) has received the most attention of circuit

designers. This is because of the fact that the CCII proves to be a versatile building block that

can be used to implement a variety of high-performance circuits which are simple to

construct. The CCII has a disadvantage that only one of the input terminals exhibits a high

input impedance (Y terminal). This disadvantage becomes evident when the CCII is required

to handle differential signals, as in the case of an instrumentation amplifier. Differential

signal processing also has the advantage of better rejection of common-mode signals. The

first step towards removing the said disadvantage came in 1989 when Pal [6] proposed a new

form of the CCII and called it the modified CCII. Pal‟s modified CCII and the resultant

DVCC II are presented in Figure 1 below.

Although enjoying differential inputs, the DVCC II of Figure 1 was not taken up very

enthusiastically by the research fraternity. The DVCC remained a neglected active building

block till 1997 when Elwan and Soliman [7] came up with a novel realization. This DVCC

implementation, shown in Figure 2, utilized the dominant monolithic integration technology

viz. CMOS.

Elwan and Soliman [7] were also the first to give a formal matrix equation (1) describing

the functionality of the five port DVCC.

[ ]

[

]

[ ]

(1)

Z

Z

Y1

Y2

X

CC II

±

+

-

Y

Z

Z

Y1

Y2

X

DVCC II

±

Figure 1. Modified Current Conveyor and its symbolic equivalent [6]

Y2 Y1 X Z1 Z2

Vb

Vb1

VSS

VDD

M2M1 M3 M4

M5 M6

M7 M8 M9 M10

M11

M12

M13

M14 M15 M16

M17 M18

Figure 2. CMOS implementation of the DVCC [7]

Linear and Non-Linear Applications of CMOS Dvcc

3

VCM

Y1Y2

VB1

VB2

VB1

VBX

VB2 VB3

X Z

VDD

VSS

M1 M2 M3 M4

M5 M6 M7 M8

M9M10

M11

M13

M12

M14

M15

M17

M16

M18

M19

M20

M21

M22 M24

M23

Figure 3. Low-Voltage Low-Power CMOS DVCC [9]

Several new and improved CMOS implementations were reported in the technical

literature after the circuit of Figure 2. These included a DVCC realization based on Quasi-

Floating Gate (QFG) transistors proposed by Moradzadeh and Azhari [8]. Mahmoud

presented a low-voltage low-power implementation of the DVCC which is reproduced in

Figure 3 for reference [9].

Another popular version of the DVCC, which is a slightly modified form of the circuit of

Figure 2, has been used in [10] and is reproduced in Figure 4. Other noteworthy DVCC

realizations can be attributed to Hu et al. [11], Siripruchyanan and Jaikla [12] (a bipolar

implementation) and, more recently, Hassan and Mahmoud [13].

Y1Y2

VBB

M1 M4M2 M3

M5 M6

M9 M10

M7

M11

M8

M12

XZ+

VSS

VDD

M16

Z-

M17 M18

M15M14M13

Figure 4. Improved CMOS implementation of DVCC [10]


4

Y1

Y2

Z+

Z-

DVCC

X

Y2

Y1

IIN Iout1

Iout2

Figure 5. DVCC based basic current processing block [15]

II. SURVEY OF SOME EXISTING DVCC BASED CIRCUITS

The literature is replete with both linear and non-linear applications of DVCC [6-27].

Amongst the linear DVCC-based circuits that are reported, some are discussed here.

A. Amplifiers

A voltage-mode amplifier using DVCC was presented in [14]. Another amplifier circuit

that needs a mention is the versatile building block shown in Figure 5 [15]. It can be made to

work as a current-mode summing amplifier by proper selection of the admittances Y1 and Y2.

For instance, if the admittances are Y1 = 1/ R1 and Y2 = 1/ R2, the current-mode amplifier can

be realized with a gain equal to (R1/ R2).

A DVCC-based instrumentation amplifier was presented in [13] and is reproduced in

Figure 6 below. The circuit takes a differential input voltage, multiplies it with a gain, and

produces a single-ended output voltage. The relation between the output voltage and the

differential input voltage was shown to be

(2)

Y2

Y1

DVCC

X

Z+

R1

VOUT

R2

+

-VIN-DIFF

Figure 6. DVCC based Instrumentation Amplifier [13]


5

B. Integrators

A voltage-mode integrator using DVCC was presented in [14]. A current-mode integrator

circuit was obtained from the basic current processing block, shown in Figure 6, in [15] by a

judicious selection of the admittances Y1 and Y2. By choosing Y1 = sC1 and Y2 = 1/ R2 and

the transfer function was shown to be

(3)

C. Filters with Bi-Linear Transfer Functions

Another application area where the DVCC has been put to good use is the design of

voltage-mode and current-mode analog filters. First, a current-mode first-order all-pass filter

section is discussed. The circuit of Figure 7, attributed to Minaei [16], presents the DVCC

based first order AP (all-pass) filter. The advantage of using a DVCC can be understood by

the fact that due to the differential input at Y-terminals, a very simple all-pass filter

realization is obtained. Another noteworthy point is that the circuit is cascadable and employs

only grounded passive components. This makes the circuit attractive for monolithic

integration. Selecting R2=2R1, the current transfer function for the AP filter of Figure 7 [16] is

given in (4) below. By using (4), the gain and phase response of the filter were shown to be

those given in (5) and (6).

A current-mode all-pass section using a new „modified‟ DVCC was reported in [17] and

is reproduced in Figure 8. The DVCC utilized in this case had two Z+ outputs and no Z-

output. Moreover, one of the Z+ stages had a gain of 2. This was achieved by tweaking the

aspect ratios of the transistors comprising the second Z+ stage.

(4)

(5)

( ) ( ) (6)

Y1

Y2

Z+

Z-

DVCC

X

Iout

Iout

R1

R2C1

IIN

IIN

R3

Figure 7. First-Order Dual-Input All-Pass Filter using a Single DVCC [16]


6

Y2 Z+

2Z+

DVCC

X

RX

Iout

Iin

Iin

C

Figure 8. Current-mode All-Pass Filter using a Modified DVCC [17]

Assuming ideal DVCC operation, the current transfer gain for the circuit of Figure 8 [17]

is reproduced below.

(

⁄ )

( ⁄ ) (7)

Equation (7) is the standard transfer function of an all-pass filter. The circuit of Figure 8

therefore provides a constant unity gain at all frequencies and frequency dependent phase

with a value

( ) ( ) (8)

The circuit [17] with only two passive components, both grounded, is quite compact and

ideal for monolithic implementation. Besides, the circuit also exhibits high output impedance

making it suitable for current-mode cascading.

Another class of first order single voltage input filter circuits based on DVCC that

provide five independent outputs simultaneously; three of which are voltage outputs and three

as current outputs were proposed by Maheshwari [18]. Shown in Figure 9, the DVCC based

circuit takes a single voltage-mode input and is able to provide low-pass, high-pass and all-

pass filter functions both in the voltage- as well as current-mode.

Y1

Y2

Z+

Z-

DVCC

X

Y1

Y2 Z+

Z-

DVCC

XC

R

VIN

VLP

RL

VAP

IAP

ILPVHP

RL

IHP

Figure 9. DVCC based first order filter with single input and six outputs [18]


7

The various voltage-mode and current-mode filter transfer functions for the circuit [18]

are reproduced below.

(9)

(10)

(11)

[

] (12)

[

] (13)

[

] (14)

It is to be noted that six first order responses (9 – 14) are simultaneously obtained from

the circuit with a total active and passive component count of five, which makes the circuit

unique till date.

A voltage-mode all-pass section was also reported in [19] and is reproduced in Figure 10.

The canonical first-order all-pass section employed a single grounded capacitor and two

DVCCs, whose RX was tuned through bias voltage, thereby ensuring a resistor-less

realization. The idea of controlling the RX for DVCC was in fact first introduced in this work

[19]. This development proved to be a significant one for realizing active-C tunable networks

using DVCC.

Y1

Y2

Z+

Z-

DVCC

X

Y1

Y2

Z+

Z-

DVCC

X

VOUT

VIN

C

Figure 10. The first voltage-controlled voltage-mode first-order all-pass section using DVCC[19]


8

The transfer function of the circuit [19] was shown to be

(15)

As can be seen from (15), the circuit of Figure 10 indeed acts as a voltage-mode all-pass

filter and exhibits a gain of unity for all frequencies (ideally) and has a frequency dependent

phase characteristics given by

( ) ( ) (16)

D. Biquadratic Filters

Elwan and Soliman [7] proposed a CMOS continuous time filter for analog signal

processing in the current-mode domain. The electronically tunable band-pass filter, shown in

Figure 11 [7], was shown to have the following transfer function

(17)

Where

( ) (18)

( ) (19)

The center frequency and quality factor of the current-mode band-pass filter of Figure 11

were given as

√

(20)

√

(21)

Equations (20) and (21) suggest that the circuit does not exhibit non-interactive control

over ωo and Q, which is a serious drawback.

Next, the differential all-pass and notch filter circuit is discussed [20]. From Figure 12, it

can be seen that the circuit accepts differential voltage-mode input (Vid) and provides a

differential output (Vod). By selecting Rz1 = Rz2 = Rz and Cz1 = Cz2 = Cz, the differential-mode

gain of the circuit of Figure 12 [20] can be written as

( )

( ) (22)


9

Y1

Y2

Z+

Z-

DVCC

X Y1

Y2 Z+

Z-

DVCC

X

C1

VC1

IBP

VC2

C2

IIN

Figure 11. DVCC based Current-mode MOSFET-C Bandpass Filter [7]

Y1

Y2 Z+

Z-

DVCC

X

Vi1

Vi2

Vo1

Vo2

RX

CX

+

-

+

-Vid Vod

Rz1

Cz1

Cz2

Rz2

Figure 12. Differential Voltage-mode All-Pass and Notch Filter Circuit [20]

From (14), it is evident that voltage-mode all-pass filter function is realized by choosing

CX = 2CZ and RZ = 2RX. Similarly, the notch can be realized by selecting CX = CZ and RZ =

RX.

Chen and Shen proposed a three-input, five-output, universal capacitor-grounded voltage-

mode filter using DVCCs [21]. The circuit, shown in Figure 13, employs two differential

voltage current conveyors as active elements together with two grounded capacitors and four

resistors as passive elements. The proposed configuration can be used as either in the single-

input five-output or three-input two-output configuration. The circuit was shown to

simultaneously realize five different generic filtering signals: lowpass, bandpass, highpass,

bandreject, and allpass while enjoying the advantage of all grounded capacitors. Depending

on the status of the three biquad input voltages, Vi1, Vi2, and Vi3, different filter functions are

reproduced in the table below.

Table I. Obtaining different filter functions from the circuit of figure 13

Node Voltages Filter Function Vi1 Vi2 Vi3 Vout

Low-Pass Vin 0 0 Vo2

Band-Pass 0 Vin 0 Vo3

High-Pass 0 0 Vin Vo3

Band-Stop Vin 0 0 Vo3

All-Pass Vin Vin 0 Vo3


10

Y1

Y2

Z+

Z-

DVCC

X

Y1

Y2Z+

Z-

DVCC

X

Vo4

Vi2

R1

Vo5

R4

C1

Vi3

R3

C2

Vi1

Vo3

Vo2

Vo1

Figure 13. DVCC based voltage-mode biquad [21]

Y1

Y2

Z+

Z-

DVCC

X Y1

Y2 Z+

Z-

DVCC

X

Y1

Y2

Z+

Z-

DVCC

X

ILP

IBPIHP

IIN

R2 R3 R4

R1 C1 C2

Figure 14. Current-mode KHN biquad based on DVCC [15]

A multi-functional filter based on the Kerwin-Huelsman-Newcomb (KHN) topology and

utilizing DVCCs was also proposed by Ibrahim, Minaei and Kuntman [15]. The circuit,

shown in Figure 14, operates in the current-mode domain and employs three DVCCs as active

elements together with two capacitors and four resistors as passive elements, all of which are

grounded. The circuit simultaneously provides the three basic filter functions, viz. band-pass,

high-pass and low-pass functions. The notch and all-pass functions can be obtained by

connecting appropriate output currents directly without using additional active elements.

The circuit of Figure 14 was shown to yield the following current-mode transfer

functions

( )

(

)

(23)

( )

(

)

(24)

( )

(

)

(25)


11

The natural angular frequency ωo and quality factor of the filters can be expressed as

√

(26)

√

(27)

It can be seen that the three basic filter functions (HP, BP and LP) are obtained

simultaneously, all at high impedance outputs. Also, for R1=R2, a notch function can be

realized by connecting high-pass and low-pass outputs, i.e. by adding the corresponding

currents without using additional active elements. Moreover, for R1=R2=R3, an all-pass

function can be obtained by connecting high-pass, low-pass and band-pass outputs. Equations

(26) and (27) further show that non-interactive control of ωo and Q is only possible by

varying resistance ratio. Other realizations of the KHN biquad using three or four DVCCs

appear in [22].

Another universal biquadratic filter that needs to be mentioned is the one proposed by

Horng et al. [23]. The circuit, which is reproduced in Figure 15, employs three DVCCs and

six passive components to yield five voltage-mode filter functions viz. low-pass, high-pass,

band-pass, band-stop and all-pass simultaneously. An attractive feature of the circuit from

monolithic implementation point of view is that all the capacitors are grounded.

The transfer functions, resonance angular frequency and quality factor for the circuit of

Figure 15 are reproduced below.

(28)

(29)

(30)

(31)

(32)

√

(33)

√

(34)

Equations (33) and (34) show that ωo and Q can be tuned independent of each other,

thereby making the circuit useful.


12

Y2

Y1

DVCC

X

Z+

Y2

Y1

DVCC

X

Z+

Y2

Y1

DVCC

X

Z+

R2R1

C1 C2

Vo1 Vo2

Vo3

VIN

Vo5

R3 R4

Vo4

Figure 15. Voltage-mode Universal Biquadratic Filter employing DVCCs [23]

E. Sinusoidal Oscillators and Function Generators

Another class of DVCC-based analog circuit applications that will be considered includes

sinusoidal oscillators and function generators. The first such circuit is the four phase clock

generator [19]. As can be seen in Figure 16, an all-pass section is used to provide a 90-degree

phase shift to an input sinusoidal signal and then two DVCC based comparators are used to

obtain four distinct square waveforms in phase quadrature. The use of DVCC as comparator

was shown for the first time in [19], which led to further new applications, as would also be

shown later in this chapter.

Y1

Y2

Z+

Z-

DVCC

X

Y1

Y2

Z+

Z-

DVCC

X

V1

V2

VIN

APS

V3

V4

Figure 16. Four phase clock generator using an all-pass section [19]

Y2

Y1

DVCC

X

Z+

R3

R1

Z-

R2

C1

C2

Iout

Figure 17. Current-mode Single Resistance Controlled Oscillator using DVCC [24]


13

Y1

Y2

Z+

Z-

DVCC

X

Y1

Y2

Z+

Z-

DVCC

X

IOUT

Y4Y2 Y3Y1

VOUT

Figure 18. Single element controlled mixed-mode sinusoidal oscillator based on DVCC [25]

A DVCC-based circuit which belongs to the category of single element controlled

oscillators is discussed next. The circuit, attributed to Gupta and Senani [24] and shown in

Figure 17, employs only a single DVCC to provide a current-mode sinusoidal output which

can be controlled by varying a single resistance. It needs to be mentioned that the particular

circuit was the first single-resistance controlled current-mode oscillator using DVCC which

employed all grounded capacitors.

The frequency and condition of oscillation for the circuit are reproduced below.

FO:

√

(35)

CO: [

] (36)

Since R1 does not feature in FO above, the CO may be set by varying R1 and then FO can

independently be selected by choosing a suitable value of R2.

Another second order sinusoidal oscillator based on the DVCC is discussed next. By

proper selection of the admittances Y1 through Y4, eight different oscillator structures were

obtained from the circuit shown in Figure 18 [25]. For instance, by setting

a mixed-mode sinusoidal oscillator is obtained where the frequency and condition of

oscillation are given as

FO:

√

(37)

CO: (38)

An evident advantage of the above circuit is that the FO (frequency of oscillation) can be

adjusted by tuning a single element (capacitor or resistor) without disturbing the CO

(condition of oscillation) while the CO can be adjusted by tuning another element (resistor or

capacitor) without disturbing the FO thereby ensuring independent amplitude and frequency

control of oscillations.


14

Y1

Y2

Z+

Z+

DVCC

X

Y1

Y2

Z+

Z-

DVCC

X

R2

C2C3

R1

C1

R3

Figure 19. Voltage-mode fully-differential sinusoidal oscillator [26]

Next, a fully differential sinusoidal oscillator is discussed. Khan and Beg [26] proposed a

DVCC based oscillator capable of producing a fully differential sinusoidal waveform. A

voltage-mode fully-differential all-pass filter section is used to realize a voltage-mode fully-

differential sinusoidal quadrature oscillator by cascading it with a differential integrator as

shown in Figure 19. The use of one floating capacitor poses integration problems. Moreover,

the R1-C1 series combination connected at the X-terminal limits the frequency generated by

the circuit.

For the circuit of Figure 19, the frequency and condition of oscillation were found out to

be [26]

FO:

√

(39)

CO: (40)

Unlike the class of second order oscillators, a third order quadrature oscillator was

recently proposed as an application of a new first order topology by Maheshwari [13]. The

DVCC paper [13] was first of its kind, encompassing analog circuit applications ranging from

first-order to third-order. It is to be noted that Maheshwari [13, 27] was first to propose third

order quadrature oscillators based on DVCC. The circuit in [13] provided four quadrature

voltage outputs, whereas the circuit in [27] was shown to provide seven distinct voltage and

current outputs. The idea of voltage controlled DVCC [19] was further utilized in [27] to

realize a voltage controlled quadrature oscillator of third order.

III. PROPOSED CIRCUITS

This section presents some new applications of the DVCC. Some neural circuits based on

the DVCC are proposed which find applications in solving simultaneous linear equations,

linear programming problems and quadratic programming problems. Further, new

applications of the DVCC in realizing digital logic functions like AND/NAND and OR/NOR

are also explored.


15

A. Linear Equation Solver

The quest for methods to solve systems of linear equations has been going on for almost

two millennia [28-30]. The state-of-art linear equation solvers use software algorithms.

However, since most computer algorithms are sequential, the time to arrive at the solution is

governed by the problem size and the hardware on which the software is run. For real-time

systems, such a solution time of the order of milliseconds may not be appropriate. For

improved solution time, the inherent parallel processing of neural networks may be utilized

[30]. This section presents a neural circuit employing non-linear feedback to solve systems of

linear equations.

Let the set of simultaneous linear equations to be solved be

( ) (41)

( ) (42)

Where V1 and V2 are the two variables and a11, a12, a21, a22, b1 and b2 are constants.

Figure 20 shows the proposed hardware for solving a set of 2 simultaneous linear equations in

2 variables.

As can be seen from Figure 20, individual equations from the set of equations to be

solved are passed through non-linear synapses. CMOS Differential Voltage Current

Conveyors (DVCCs) are used as voltage-mode comparators and opamps are utilized to

emulate the operation of neurons. The DVCC is able to provide multiple current outputs

simultaneously, at the Z+ ports, which are scaled suitably to generate weighted currents at the

input of the neurons. For this purpose, use is made of the W/L ratios of the MOSFETs

employed in the output stage of the DVCC. Rp1 & Rp2 are the parasitic resistances and Cp1 &

Cp2 are the parasitic capacitance that present themselves at the inputs of the Opamps used in

the realization of neurons. These parasitic components are included to model the dynamic

nature of the Opamps. For the DVCC, we know that

( ) (43)

( ) (44)

For the DVCC, comparator action can be achieved by directly grounding the X-terminal.

The use of DVCC as a voltage-mode comparator was first given in [19]. For such a case, the

current in the X-port will saturate and can be written as

( ) ( ) (45)

By virtue of DVCC action, this current will be transferred to the Z+ ports. For DVCC-1,

the output currents at the Z+ ports can be written as

( ) ( ) (46)


16

( ) ( ) (47)

Where and are the scaling factors introduced during the current copying process

from the X-port to the Z-port. Ideally, these factors are unity. Similarly, for the second

comparator, we may write

( ) ( ) (48)

( ) ( ) (49)

Where is the open-loop gain of the comparator (practically very high) and is the

maximum (saturated) current at the output of the comparator. Writing node equation for the

nodes „A‟ and „B‟ in Figure 20, we get

( – ) ( – )

( )

( ) ( )

( )

Moreover, it is known that in the case of symmetric connections, the equations of motion

for this network of analog processing units corresponds to a stable system, in which the

output voltages of all amplifiers become constant [31]. Also, the stable states of the network

are the local minima of a quantity known as the computational energy „E‟ of the system (more

commonly referred to as „Lyapunov function‟ or „Energy Function‟) [32].

b1 x1

b2

V2Rp2 Cp2

B

V1Rp1 Cp1

As11IX1

Z1+

Z2+

X

X

Y2

Y1

Y2

Y1

DVCC-1

DVCC-2

a21V1 + a22V2

a11V1 + a12V2s21IX1

s12IX2

s22IX2

Z1+

Z2+

Figure 20. Proposed DVCC-based CMOS compatible circuit to solve a system of simultaneous linear

equations in 2 variables


17

For the network of Figure 20, it can be shown that the Energy Function „E‟ of is related

to the voltages V1 and V2 by

( ) ( ) [

] (52)

( ) ( ) [

] (53)

Also, if „E‟ is the Energy function, it must satisfy the following conditions for stability

[33].

(54)

And

(55)

Comparing equations (50) & (52), and (51) & (53) according to (54) and (55), we get

Although Figure 20 shows the proposed circuit as applicable to a system of simultaneous

linear equations in two variables, the technique can be readily extended to solve n-

simultaneous linear equations in n- variables. Moreover, for obtaining weighted (scaled)

current outputs use was made of the (W/L) ratios of the transistors comprising the Z+ stages

of the DVCCs used in Figure 20. This results in a particular circuit implementation „hard-

wired‟ to solve a given set of equations. Methods to scale the currents by a mechanism

external to the circuit need to be investigated to impart flexibility to the circuit.

B. Linear Programming Circuit

In a linear programming problem (LPP), a linear function is optimized subject to certain

linear constraints [34-36]. Linear programming can be applied to various fields of study. It is

used most extensively in business and economics, but can also be utilized for some

engineering problems. Industries that use linear programming models include transportation,

energy, telecommunications, and manufacturing. It has proved useful in modeling diverse

types of problems in planning, routing, scheduling, assignment, and design.


(56)

Subject to


18

(57)

(58)

Where V1, and V2 are the variables and aij, bi and cij are constants. The proposed neural-

network based circuit to minimize (56) subject to (57, 58) is presented in Figure 21. As can be

seen from Figure 21, individual equations from the set of constraints are passed through non-

linear synapses which are realized using DVCC-based unipolar comparators. The outputs of

the comparators are fed to neurons having weighted inputs. These weighted neurons are

realized by using opamps where the scaled currents coming from various comparators act as

weights. The parasitic components that need to be included to model the dynamic nature of

the opamps are also considered (as was done for the previous circuit) and are shown in Figure

21.

Node equation for nodes „A‟ and „B‟ give the equations of motions of the first and second

neurons as

[

] [

] (59)

Where, and represent the currents and after passing through

diodes as shown in Figure 21 and

(60)

And,

[

] [

] (61)



(62)

In equations (59) and (61), ui is the internal state of the i-th neuron and is the current

scaling factor at the j-th output of i-th DVCC. As is shown later in this section, these weights

are governed by the constraint inequalities (57, 58). Using (46, 47) in (59) and (61) results in

(63) and (64) given below.

[ ( – ) ]

[ (

– ) ] [

] [

] (63)


19

s11IX1’

b1

x1

s11Ix1

Z+

1

XY2

Y1

DVCC-1

a11V1+ a12V2

Z+

2

s12Ix1

b2

x1s21Ix2

Z+

1

XY2

Y1

DVCC-2

a21V1+ a22V2

Z+

2

s22Ix2

+

_

V1

+_

Rp2C2V2

Rp1 C1

R1

R2

A

B

P1

P2

s21IX2’

s22IX2’

s12IX1’

Figure 21. Proposed neural network for solving a linear programming problem in 2 variables subject to

2 linear constraints

[ ( – ) ]

[ (

– ) ] [

] [

] (64)

Moreover, it can be shown that the network of Figure 21 may be associated with an

„energy‟ function of the form

( – )

( – )

( )

∫

∫

(65)


( ) ( )

[

] (66)

Also, if „E‟ is the Energy Function, it must satisfy the following condition for stability

[33].

(67)

Where „K‟ is a constant of proportionality and has the dimensions of resistance.



20

⁄

⁄ (68)

(69)

Analysis on similar lines can be performed to obtain the values of the synaptic weights

for the second neuron and are presented in (70) and (71) below.

⁄

⁄ (70)

(71)

The values of the current scaling factors (sij) of Figure 21 can be calculated by choosing a

suitable value of „K‟ and then using (68, 70).

C. Quadratic Programming Circuit

Quadratic Programming problems are very important in the field of optimization [37-39].

They arise in many applications such as constrained least mean square estimation and the

classical newsvendor problem. Besides its wide applications, quadratic programming is also

forms a basis for solving some general nonlinear programming problems as most higher order

problems can usually be approximated by a second order function. Traditional methods for

solving quadratic programming problems typically involve an iterative process, but long

computational time limits their usage. There is an alternative approach to solution of this

problem. It is to exploit the artificial neural networks which can be considered as an analog


Let the quadratic function to be minimized be

(72)

Subject to

(73)

(74)

Where V1, and V2 are the variables and aij, bi and cij are constants. The proposed neural-

network based circuit to minimize (72) subject to (73, 74) is presented in Figure 22.

As was done in the case of linear programming circuit, individual equations from the set

of constraints are passed through non-linear synapses which are realized using DVCC-based

unipolar comparators. The outputs of the comparators are fed to neurons having weighted

inputs. These weighted neurons are realized by using opamps where the scaled currents

coming from various comparators act as weights. Weighing is also provided by the self-


21

feedback resistances (R11 & R22 for neurons 1 & 2 respectively) and the cross-feedback

resistances (R21 and R12).

Node equation for nodes „A‟ and „B‟ give the equations of motions of the first and second

neurons as

[

] [

] (75)



(76)

And,

[

] [

] (77)



(78)

b1

x1s11Ix

Z+

1

XY2

Y1

DVCC-1

a11V1+ a12V2

Z+

2

s12Ix

b2

x1s21Ix

Z+

1

XY2

Y1

DVCC-2

a21V1+ a22V2

Z+

2

s22Ix

+

_V1

+_

Rp2Cp2

V2

Rp1 Cp1

R11

R22

R21

R12

A

B

s11IX’

s12IX’

s21IX’

s22IX’

Figure 22. Proposed circuit to solve quadratic programming problem in 2-variables with 2 linear

constraints


22

In equations (78) and (80) ui is the internal state of the i-th neuron and is the current

scaling factor at the j-th output of i-th DVCC. As is shown later in this section, these weights

are governed by the constraint inequalities (73, 74). Using (46, 47) in (75) and (77) results in

(79) and (80) given below.

[ ( – ) ]

[ (

– ) ] [

] [

] (79)

[ ( – ) ]

[ (

– ) ] [

] [

] (80)

Moreover, it can be shown that the network in Figure 22 can be associated with an

Energy Function „E‟ of the form

( – )

( – )

( )

∫

∫

(81)


( )

(

) [

] (82)

Also, if „E‟ is the Energy Function, it must satisfy the following condition for stability

[33].

(83)

Where „K‟ is a constant of proportionality and has the dimensions of resistance.


(84)

(85)

The values of the current scaling factors (sij) of Figure 22 can be easily calculated by

setting the value of „K‟ to be equal to ⁄ and then using (84). The values of the weight

resistances connected to the first neuron, R11 and R21, can be obtained by using (85). Analysis


23

on similar lines can be performed to obtain the values of the synaptic weights for the second

neuron.

D. Implementation of Logic Gates Using DVCC

Another application area where the DVCC can be put to use is Digital Logic Gates.

Although implementations of various digital logic gates are abundant in various technologies

including CMOS, the use of analog building blocks like the DVCC has never been considered

before. The circuit for 2-input AND/NAND gate, presented in Figure 23, is a first attempt to

showcase the full potential of the DVCC and is by no means a comparison with the existing

implementations, which employ fewer transistors.

The working of the AND gate of Figure 23 can be understood as follows. As long as V2

is low, transistor M1 remains off leading to IX = 0 (irrespective of the voltage applied at Y1

terminal) which causes IZ and consequently VAND to be zero. When V2 becomes high,

transistor M1 conducts and the current IX will be governed by the presence or absence of V1.

Under such a condition, for the case of V1 being low (zero), IX will be zero and consequently,

VAND is low. For the case when V1 is high, the output voltage VAND will be high. The input

and output for the AND gate are related as

( )

Similarly, the circuit for OR/NOR gates can be obtained if the Y3 terminal of the DVCC

is also utilized [13]. An analysis similar to that done for the AND/NAND circuit shows that

the circuit of Figure 24 does indeed works as an OR/NOR gate. It may be mentioned that

VOR/VNOR may be needed to be passed through an additional comparator if a binary digital

output is required.

Y2

Y1

DVCC

X

Z+V1

V2

R1

VAND

M1

R1

VNANDZ-

Figure 23. AND/NAND gate realization using DVCC

Figure 24. OR/NOR gate realization using DVCC

Y2

Y1

DVCC

X

Z+V1

V2

R1

VOR

M1

R1

VNOR

M2V1

Y3 Z-


24

IV. SIMULATION RESULTS

This section deals with the simulation results for the proposed circuits. Simulations were

carried out using PSPICE program. The circuit for DVCC shown in Figure 4 was utilized and

standard 0.5 micron CMOS parameters were used for simulation purposes. For the opamp,

use was made of the LMC7101A CMOS opamp from National Semiconductor. The sub-

circuit file for this opamp is available in Orcad Model Library.

The proposed circuit for solving linear equations was tested for solving the following set

of 2 simultaneous linear equations in 2 variables.

[

] [ ] [

] ( )

For the above set of equations, the required weighted currents that need to be generated

are given as: s11 = 2, s12 = 2, s21 = 1 and s22 = 1. Scaling of the Z-port currents was done by

varying the aspect ratios of the transistors in the output stage of the DVCC. The circuit used

for PSPICE simulation is presented in Figure 25. As can be seen, some additional circuitry is

required to generate the proper input voltages of the comparators [30]. The values of the

different resistances shown in Figure 23 are: Re11 = 2K; Re12 = 4K; Re13 = 4K; Re21 = 3K; Re22

= 3K; Re23 = 3K.

PSPICE simulations were performed for the chosen set of equations and the results are

given in Figure 26. As can be verified from the figure, the simulation results are found to

agree exactly with the algebraic solution.

Figure 25. The proposed circuit applied to solve 2 linear equations in 2 variables

b1 x1

b2

Re21

Re22

Re23

V2

Re11

Re12

Re13

V1

Z+

Z+

X

X

Z+

Z+

Y2

Y1

Y2

Y1

DVCC-1

DVCC-2


25

Figure 26. PSPICE simulation results of the circuit to solve (86)

Next, the proposed network for solving linear programming problems was tested using

PSPICE simulation. The circuit was used to minimize the objective function

(87)

subject to

The values of resistances acting as the weights on the neurons are obtained from (69, 71).

Towards that end, first the value of was computed for the DVCC which was found to be

1.091 milli-mhos. Choosing ⁄ allows us to fix the current scaling factors as given

below:

Routine mathematical analysis of (87, 88) yields: = 0, = 1. The results of PSPICE

simulation are presented in Figure 27. From the plots of the neuron output voltages, it can be

seen that V(1) = 0 V and V(2) = 1 V which correspond exactly to the algebraic solution

thereby confirming the validity of the approach.

(88)


26

Figure 27. Simulation results for the proposed circuit for solving linear programming problems (87, 88)

The proposed neural circuit for solving quadratic programming problems was tested by

minimizing the objective function

(89)

subject to

The values of resistances acting as the weights on the neurons are obtained from (85).

The corresponding values of the weight resistances are found to be: R11 =305.53Ω; R21

=229.14Ω; R12 =229.14Ω; R22 =183.318Ω. The values of the current scaling factors were

found out to be: s11 = 1; s12 = -1; s21 = 1 ; s22 = 1.

Routine mathematical analysis of (89, 90) yields: = -0.584, = 0.416. The results of

PSPICE simulation are presented in Figure 28. From the plots of the neuron output voltages,

it can be seen that V(1) = -0.58 V and V(2) = 0.41 V which are very near to the algebraic

solution thereby confirming the validity of the approach.

Next, the AND gate of Figure 23 was verified. The results of simulation are presented in

Figure 29 where the voltages V1 and V2 are the inputs to the AND gate and voltage V4 is the

AND-ed output.

(90)


27

Figure 28. Simulation results for the chosen 2 – variable problem (89, 90)

Figure 29. Simulation results for the proposed AND gate using DVCC

An important point that needs to be discussed is the VLSI implementability issue of the

existing and proposed CMOS circuits. It is well known that in standard CMOS technology,


28

processing variations prohibit the fabrication of individual resistances with precise values. It

is, however, possible to fabricate a pair of resistors having the same value, although the exact

value may not necessarily be predicted. Considering this fact, one way of implementing

resistances in CMOS analog circuits is to tweak the design such that the critical (performance

deciding) factors are governed by resistance-ratios [40]. Wherever this is not possible, use

may be made of MOS-based resistors. The function of a resistor may be obtained using

MOSFETs to realize voltage-controlled transconductances or by linearizing the transistor‟s

region of operation in the triode region [41].

V. CONCLUSION

This chapter was based on circuit applications of differential voltage current conveyor in

CMOS technology. A survey of existing linear and non-linear circuits was presented. Some

new applications of DVCC were also proposed. The chapter explores a wide variety of analog

and a few digital applications and is expected to both provide a deep insight into the subject

and also further enhance the existing knowledge.

REFERENCES

[1] Smith, K. C. & Sedra, A. S. (1968). “The current-conveyor-a new circuit building

block,” IEEE Proc., Vol. 56, Pages 1368-69.

[2] Maheshwari, S. & Khan, I. A. (2005). “Current controlled third order quadrature

oscillator,” IET Circuits Devices Syst., Vol. 152, No. 6, Pages 605-607.

[3] Khan, I. A. & Ahmed, M. T. (1986). “Electronically tunable first-order OTA-capacitor

filter sections,” Int. J. of Electronics, Vol. 61, No. 2, Pages 233-237, August.

[4] Biolek, D. (2003). “CDTA–Building block for current-mode analog signal processing”,

Proc. ECCTD Poland, Pages 397-400.

[5] Minaei, S. & Yuce, E. (2008). “A new full-wave rectifier circuit employing single dual-

X current conveyor,” Int. J. of Electronics, Vol. 95, No. 8, Pages 777-784, August.

[6] Pal, K. (1989). “Modified current conveyors and their applications,” Microelectronics

Journal, Vol. 20, No. 4, Pages 37-40.

[7] Elwan, H. O. & Soliman, A. M. (1997). “Novel CMOS differential voltage current

conveyor and its applications,” IEE Proc. Circuits Devices Syst., Vol. 144, No. 3, Pages

195-200, Jun.

[8] Moradzadeh, H. & Azhari, S. J. (2008). “High performance low-voltage QFG-based

DVCC and a novel fully differential SC integrator based on it,” IEICE Electronics

Express, Vol 5, No. 23, Pages 1017-1023, Dec.

[9] Mahmoud, S. A. (2008). “Low Voltage Wide Range CMOS Differential Voltage

Current Conveyor and its Applications,” Contemporary Engineering Sciences, Vol. 1,

No. 3, Pages 105-126.

[10] Minaei, S. & Ibrahim, M. A. (2005). “General configuration for realizing current-mode

first-order all-pass filter using DVCC,” Int. J. of Electronics, Vol. 92, No. 6, Pages 347-

356.


29

[11] Hu, J., Xu, T., Zhang, W. & Xia, Y. (2005). “A CMOS Rail-to-Rail Differential

Voltage Current Conveyor and its Applications,” IEEE Proc. ICCCAS, Vol. 2, Pages

1079-1083.

[12] Siripruchyanan, M. & Jaikla, W. (2007). “Floating Capacitance Multiplier Using

DVCC and CCCIIs,” Proc. Int. Symp. on Comm. and Info. Tech. (ISCIT '07).

[13] Hassan, T. M. & Mahmoud, S. A. (2010). “New CMOS DVCC realization and

applications to instrumentation amplifier and active-RC filters,” Int. J. Electronics and

Communications, Vol. 64, Pages 47-55.

[14] Maheshwari, S. (2009). “Analogue signal processing applications using a new circuit

topology,” IET Circuits Devices Syst., Vol.3, No.3, Pages 106-115.

[15] Ibrahim, M. A., Minaei, S. & Kuntman, H. (2005). “A 22.5 MHz current-mode KHN-

biquad using differential voltage current conveyor and grounded passive elements,” Int.

J. Electronics and Communications, Vol. 59, No. 5, Pages 311-318, Jul.

[16] Minaei, S. & Temizyurek, C. (2003). “Dual Input All-Pass Filter using DVCC,” Proc.

Int. Symposium on Signals, Circuits and Systems (SCS), Vol. 2, Pages 477-480.

[17] Maheshwari, S. (2007). “Novel cascadable current-mode first order all-pass sections,”

Int. J. Electronics, Vol. 94, No. 11, Pages 995-1003, Nov.

[18] Maheshwari, S. (2008). “CMOS Compatible Single Input Five/Six Outputs First Order

Filter Circuits,” J. Active Passive Elec. Dev., Vol. 3, Pages 125-133.

[19] Maheshwari, S. (2008). “A Canonical Voltage-Controlled VM-APS with a Grounded

Capacitor,” Circuits, Systems, and Signal Processing, Vol. 27, No. 1, Pages 123-132,

Feb.

[20] Ibrahim, M. A., Minaei, S. & Kuntman, H. (2006). “DVCC based differential-mode all-

pass and notch filters with high CMRR,” Int. J. Electronics, Vol. 9, No. 4, Pages 231-

240, Apr.

[21] Chen, H. P. & Shen, S. S. (2007). “A Versatile Universal Capacitor-Grounded Voltage-

Mode Filter Using DVCCs,” ETRI Journal, Vol. 29, No. 4, Pages 470-476, Aug.

[22] Soliman, A. M. (2008). “Generation and classification of Kerwin–Huelsman–Newcomb

circuits using the DVCC,” Int. J. Circ. Theor. and Appl., Vol. 37, Pages 835-855, Jun.

[23] Jiun-Wei Horng et al, (2006). “High Input Impedance Voltage-Mode Universal

Biquadratic Filter with One Input and Five Outputs Using Current Conveyors,”

Circuits, Systems, Signal Processing, Vol. 25, No. 6, Pages 767-777.

[24] Gupta, S. S. & Senani, R. (2000). “Grounded-Capacitor Current-mode SCRO: Novel

application of DVCC,” Electronics Letters, Vol. 36, No. 3, Pages 195-196.

[25] Kumar, P., Keskin, A. U. & Pal, K. (2007). “DVCC-Based Single Element Controlled

Oscillators Using All-Grounded Components and Simultaneous Current-Voltage Mode

Outputs,” Frequenz, Vol. 61, Pages 141-144.

[26] Khan, I. A. & Beg, P. (2009). “Fully Differential Sinusoidal Quadrature Oscillator

Using CMOS DVCC,” Proc. Int. Conf. On Communication, Computer and Power

(ICCCP'09), Muscat, Pages 196-198, Feb.

[27] Maheshwari, S. (2009). “Quadrature oscillator using grounded components with current

and voltage outputs,” IET Circuits Devices Syst., Vol.3, No.4, Pages 153-160.

[28] Gustafson, J. L. (2006). “The Quest for Linear Equation Solvers and the Invention of

Electronic Digital Computing,” Proc. JVA’06, IEEE John Vincent Atanasoff 2006

International Symposium on Modern Computing.


30

[29] Xia, Y., Wang, J. & Hung, D. L. (1999). “Recurrent Neural Networks for Solving

Linear Inequalities and Equations,” IEEE Trans. Circuits and Systems–1: Fundamental

Theory and Applications, Vol. 46, No. 4, Pages 452-462, April.

[30] Ansari, M. S. & Rahman, S. A. (2009). “A Novel Current-mode Non-Linear Feedback

Neural Circuit for Solving Linear Equations,” Proc. IMPACT – 2009, Aligarh, India,

Pages 284 – 287, Available on IEEE Xplore, DOI: 10.1109/MSPCT.2009.5164189

[31] Hopfield, J. J. & Tank, D. W. (1985). “„Neural‟ computation of decisions optimization

problems,” Biological Cybernetics, Vol. 52, Pages 141-152.

[32] Tank, D. W. & Hopfield, J. J. (1986). "Simple neural optimization networks: an A/D

converter, signal decision network, and linear programming circuit," IEEE Trans. Circ.

Syst, Vol. 33, Pages 533-541.

[33] Rahman, S. A. Jayadeva, & Dutta Roy, S. C. (1999). "Neural network approach to

graph colouring," Electronics Letters, Vol. 35, Pages 1173-1175.

[34] Wang, J. (1993). “Analysis and Design of a Recurrent Neural Network for Linear

Programming”, IEEE Trans. Circuits and Systems–1: Fundamental Theory and

Applications, Vol. 40, No. 9, Pages 613-618, Sept.

[35] Xia, Y. (1995). “A New Neural Network for Solving Linear Programming Problems

and its Application,” IEEE Trans. Neural Networks, Vol.7, No.2, Pages 94-103, March.

[36] Malek, A. & Yari, A. (2005). “Primal–dual solution for the linear programming

problems using neural networks,” Applied Mathematics and Computation, Vol. 167,

Pages 198-211.

[37] Kennedy, M. P. & Chua, L. O. (1988). “Neural networks for nonlinear programming,”

IEEE Trans. Circ. Syst., Vol. 35, Pages 554-562.

[38] Wu, X. Y., Xia, Y. S., Li, J. & Chen, W. K. (1996). “A high performance neural

network for solving linear and quadratic programming problems,” IEEE Trans. Neural

Networks, Vol. 7, No. 3, Pages 643-651.

[39] Malek, A. & Alipour, M. (2007). “Numerical solution for linear and quadratic

programming problems using a recurrent neural network,” Appl. Math. Computation,

Vol. 192, Pages 27-39.

[40] Uyemura, J. P. (2006). “Chip Design for Submicron VLSI: CMOS Layout and

Simulation,” Cengage Learning, First Ed., Pages 381-382.

[41] Mahmoud, S. A., Elwan, H. O. & Soliman, A. M. (1997). “Generation of CMOS

Voltage-Controlled Floating Resistor,” Microelectronics Journal, Vol. 28, Pages 627-

640.

THE PROCEEDINGS OF IEEE SPONSORED INTERNATIONAL CONFERENCE RASIET-07

IET ALWAR (RAJ) ON 22ND

-23RD

DEC 07/VOL.2

570

A Non – Linear Feedback Neural Network for Graph

Coloring

S. J. Arif* S. A. Rahman* M. S. Ansari*

Abstract

This paper presents a hardware approach to the problem of graph coloring. In the graph coloring

problem, the task is to color, or label, the vertices such that no two adjacent vertices get the same

color. The neural network model presented also minimizes the number of colors used. This

additional capability provides for a higher efficiency of resource usage as well as having

application to other problems that map to graph coloring. Hardware and simulation results on

random and benchmark problems have been presented. Test results are compared with previous

neural network techniques for graph coloring, and it is shown that this neural network model

provides a significant reduction in the number of colors required while maintaining a high

convergence rate.

1. Introduction

The map-coloring problem, famous for the four-color theorem, is a special case of the graph-coloring problem (GCP). Graph coloring has a large number of applications such as register allocation in digital computers [1], frequency channel assignment in mobile communication [2], and layer assignment in VLSI design. In GCP, values or colors are assigned to different nodes of a graph with a condition that no two adjacent nodes of the graph should have the same color, while the number of colors should also be minimized. For proper graph coloring, the adjacency constraints should be strictly satisfied. The Hopfield Neural Network and its modifications [3,4] have been applied to solve GCP, but the implementation requires N2

neurons and at most N4 weights for solving a graph of N nodes. This leads to significant consequences in terms of the feasibility of *Department of Electronics Engineering, Aligarh Muslim University, Aligarh

HNN realization in hardware. Another feedback Neural network architecture, which uses non-linear synapses, and has a transcendental Energy function, requires N neurons and at most N2 interconnections for solving a problem with N nodes [6]. In this paper, the neural circuit presented in [6] has been modified, in order to improve the solution quality. In the proposed network, the Energy function given in [6] has been modified by eliminating those local minima that correspond to large number of colors. This is achieved by replacing the bipolar comparators of the earlier circuit with unipolar comparators. It is seen that the number of colors is reduced.

2. Proposed Network For Graph

Coloring

Fig. 1 shows the ith neuron of proposed network. In the proposed circuit, output voltages of different neurons represent the colors of different nodes. Ci and ri denotes the internal capacitance and resistance of the ith neuron respectively, u

i is the internal state and Rii is the

feedback resistance of i-th neuron. The output



-23RD

DEC 07/VOL.2

571

of other neurons Vj (j = 1,2…N) are connected to the input of ith neuron through unipolar comparators. These unipolar comparators are

realized by connecting the diodes Dij in series with the bipolar comparators CMij. (j = 1,2…N).

ri

Ci

RC/D

xi1

xij

xiN

Vi

V1

Vj

VN

Ri1

Rij

RiN

Di1

Dij

DiN

CMi1

CMij

CMiN

Fig. 1 The proposed neural network based circuit for

graph coloring

If ith

neuron is connected to jth

neuron (aij =1),

then Vj is connected to the internal state of the

ith

neuron ui through comparator CMij and a

resistance Rij. This is a constant given by

Rij = aij Rc

where

1, if ith

node is connected by an

aij = edge to j

th node

0, otherwise

The transfer characteristic of the unipolar

comparator CMij is shown in fig. 2.

Fig. 2 Transfer Characterisitcs of CMij

The output of unipolar comparator CMij is

denoted by Xij . The transfer characteristic of

this comparator can be modeled by the

following equation.

( ) 2

tanh mijm

ij

VVVVX

−−=

β (1)

( )[ ] 1tanh2

−−= ijm

ij VVV

X β (2)

Where β is the gain of comparator which is

very high (β→∞), Vm is the biasing voltage of

comparator.

The equation of motion of ith

neuron is given as

∑≠=

−+

−=+

N

ijj ij

iij

ii

ii

i

iii

R

uX

R

uV

r

u

dt

duC

,1

(3)

which transforms to

∑∑≠=≠=

+−−−=N

ijj ij

ijN

ijj ij

i

i

i

ii

i

ii

iii

R

X

R

u

r

u

R

u

R

V

dt

duC

,1,1

(4)



-23RD

DEC 07/VOL.2

572

and thus ∑≠=

+−=N

ijj ij

ij

i

i

ii

iii

R

X

R

u

R

V

dt

duC

,1

(5)

where [ ]∑≠=

++=N

ijj ijiiii RrRR ,1

1111

Therefore, Ri is equivalent to parallel

combination of all the resistances connected to

the node at internal state ui.

Similarly, we have

ij

N

ijj

ij

i

iiii

ii XW

R

uVW

dt

duC ∑

≠=

+−=,1

(6)

Where ii

iiR

W1

= , ij

ijR

W1

=

The output of ith

neuron is related to the

internal state by

( )imi uVV λtanh−=

Where λ is the gain of the amplifier, which is

very high (λ→∞)?

The given circuit may be associated with the

following Lyapunov Function.

( )[ ]2

coshln4

11

,10

2

1

imij

N

ijj

mi

V

i

i

i

N

i

iic

VVVV

VdVu

RVWE

i

−−−−= ∑∫∑≠==

ββ

(7)

The second term in equation (1) is usually

neglected for λ→∞, and thus equation (7) can

be simplified to

( )[ ]2

coshln1

4 ,1

2

1

imij

N

ijj

cijm

i

N

i

iic

VVVVWa

VVWE −−−= ∑∑

≠==

ββ

(8)

The first term on the right hand side of (8) is

quadratic which tries to minimize the number

of colors by lowering the color values. The

second term has got a negative sign. Therefore,

the energy function Ec will be minimized if

second term is maximized. This happens when

the voltages corresponding to connected nodes

in a graph are far away from each other. The

first two terms on the right hand side are

balancing each other to color a graph properly.

The last term also contributes to lowering of

number of different colors by eliminating all

those local minima in the energy function for

which node voltages were negative.

3. Application Of The Proposed

Network To Graph Coloring

Problems

The proposed network was tested for various

example graphs as well as a standard

benchmark problem for graph coloring,

myciel3.col [5]. Both hardware and software

simulation tests were undertaken. As can be

seen from table 1,the proposed network gives a

solution to al the problems tested and in all the

cases the solution is very near the chromatic

number of the graph. As an example,

simulation runs of the proposed network for

the myciel3.col benchmarking problem

returned the best solution of 5 colors which is

very near to the chromatic number for

myciel3.col, that being 4.

The performance of the proposed network was

also compared with a network proposed earlier

[6]. Table 2 gives this performance

comparison. It is evident from the table that

substantial improvements have been achieved

both in the best and the average solutions.

Table 2. Performance comparison of the proposed

and existing networks



-23RD

DEC 07/VOL.2

573

4. Conclusion

A novel neural network based architecture to

solve graph coloring problems is presented.

The circuit was realized and simulated using

SPICE for an example set of eight different

graphs including the myciel3.col benchmark

problem. Simulation tests for the proposed

circuit ascertain its superiority over its

hardware predecessors. It is observed that both

the minimum and the average number of colors

are improved for the proposed network

References [1] T. Zeitlhofer and B. Wess, A Comparison of Graph

Coloring Heuristics for Register Allocation based on

Coelescing in Interval Graphs, Proc. ISCAS 04, vol. IV,

2004.

[2] N. A. El-Fishawy,M. M. Hadhood, S. Elnoubi, and

W. El-Sersy, A modified Hopfield neural network

algorithm for cellular radio channel assignment, Proc.

TENCON 2000, vol. 2, Sept 2000.

[3] K. Jin’no, H. Taguchi, T. Yamamoto, H. Hirose,

Dynamical hysteresis neural networks for graph

coloring problem, Proceedings of ISCAS03, vol. 5, 2003.

[4] W-D. Sun, H. Tamura, Z. Tang, M. Ishii, A Method

to Solve the Four-Coloring problem by the Neural

Network with Self-Feedback, Proc. SICE Annual

Conference in Fukui, Japan, Aug, 2003.

[5] J. Lui, W. Zhong, and L. Jiao, Comments on “The

1993 DIMACS Graph Coloring Challenge” and

“Energy Function-Based Approaches to Graph

Coloring”, IEEE Trans. On Neural Networks, vol. 17,

no. 2, Mar 2006.

[6] S. A. Rahman, Jayadeva and S. C. Dutta Roy, Neural

network approach to graph colouring, Electronics

Letters, vol. 35, Jul 1999

[7] Web resource for graph coloring benchmarks,

http://mat.gsia.cmu.edu/COLOR/instances.html

Mohd. Samar Ansari received the

B.Tech. degree in Electronics

engineering from the Aligarh Muslim

University, Aligarh, India, in 2001 and

the M. Tech. degree in Electronics

engineering, with specialization in

Electronic Circuit and System Design, in 2007. He is

currently a Lecturer in the Department of Electronics

Engineering, Aligarh Muslim University, where he

teaches Electronic Devices & Circuits and

Microelectronics. His research interests include

microelectronics, linear integrated circuit applications,

neural networks, and semiconductor physics.

Syed Atiqur Rahman did B.Sc.(Engg.) in Electrical

Engineering in 1988, and M.Sc.(Engg.) in Electronics

and Communication Engineering in 1994, from Aligarh

Muslim University, Aligarh. He obtained his PhD from

Electrical Engineering Department at IIT (Delhi) in

2007. He is currently a Reader in the Department of

Electronics Engineering, Aligarh Muslim University,

where he teaches neural networks and signals &

systems. His fields of interest are Electronic Circuits and

Artificial Neural Networks.

Syed Javed Arif did B.Sc.(Engg.) in Electrical

Engineering in 1988, and M.Sc.(Engg.) in 1990, from

Aligarh Muslim University, Aligarh. He is presently

working towards his PhD from Electronics Engineering

Department, Aligarh Muslim University, Aligarh. He is



teaches Digital Electronics and Electronics Devices. His

fields of interest are Electronic Circuits and Artificial

Neural Networks.


IET ALWAR (RAJ) ON 22 -23 DEC 07/VOL.2

Solution of Simultaneous Linear Equations using Feedback

Neural Network

S. A. Rahman* M. S. Ansari*

Abstract Solving a linear system of equations is one of the most important computing problems. Existing

methods include the use of parallel computing architectures and complex software algorithms.

The time taken by the existing methods to solve a linear system of equations is not always short

and increases significantly with the number of variables. This paper presents a feedback neural

network based hardware solution to the problem of solving simultaneous linear equations. A

direct consequence of the use of neural networks is massively parallel processing and fast

convergence. This results in a faster and more compact hardware. Simulation results are

presented for a chosen set of equations and are found to agree exactly with the algebraic

solution.

I. INTRODUCTION

The solution of linear system of equations

continues to play an important role in

scientific computing. The problems to be

solved often are of very large size, so that

large computer resources or massive parallel

computer systems with distributed memory

are needed to solve them. This problem is

key to many real-time engineering

applications [1-6]. Examples are, among

others, real-time speech coding, image

processing, stochastic modeling, and

computer-aided realistic three-dimensional

image synthesis. When the problem at hand

is large, we must obtain its solution within a

reasonably short period of time. Existing

methods to solve a set of linear equations

also include Gaussian elimination and the

Cholesky decomposition.

*Department of Electronics Engineering,

Aligarh Muslim University, Aligarh

To determine approximate solutions to

general systems numerically on a computer,

the n-dimensional Newton's method may be

used.

Among the concurrent multiprocessor

computers the contemporary, relatively less

expensive systolic/wavefront arrays like

Kung’s are promising parallel architectures

[7]. These structures provide the necessary

computational power to tackle large

problems efficiently in terms of time. The

Block Data Parallel Architecture (BDPA) is

an even less expensive alternative than the

systolic/wavefront arrays for solving these

computationally intensive problems [8]. The

processing of data by blocks and the use of

commercially available processors makes

this architecture not only less expensive, but

also flexible and reconfigurable. The

architecture can be used, not only for

elementary DSP algorithms, but also for

complex algorithms usually associated with

linear algebra. However, even with the

extensive simulation tools developed for the

BDPA, there is still a fundamental need to

show the prospective power of the

architecture.



This paper presents a hardware solution to

the problem of solving simultaneous linear

equations. The approach used is to employ

neural networks on the basis of an analog

circuit [2,4,6]. The most important

advantages of neural networks are massively

parallel processing and fast convergence.

Hence, architectures based on neural

networks have many computational

advantages over the traditional algorithms

running over a parallel computing system.

The following section lays the basic problem

and the mathematical formulation on which

the development of the proposed network

will be based. Section III contains the circuit

implementation of the proposed network for

a set of sample linear equations in two

variables. SPICE simulation results of the

proposed circuit are also presented. Some

concluding remarks are presented in Section

IV.

II. PROPOSED NEURAL NETWORK

BASED CIRCUIT

Let the simultaneous linear equations to be

solved be

𝑎11𝑉1 + 𝑎12𝑉2 − 𝑏1 = 0

𝑎21𝑉1 + 𝑎22𝑉2 − 𝑏2 = 0

Where V1 and V2 are the two variables and

a11, a12, a21, a22, b1 and b2 are constants.

The proposed neural-network based circuit

to solve the system of equations given above

is presented in fig. 1.

As can be seen from fig. 1, individual

equations from the set of equations to be

solved are passed through a non-linear

synapse. This non-linear synapse is in effect

a comparator which is realized using

Opamps. The outputs of the non-linear

synapses are fed to weighted neurons. These

weighted neurons are again realized by

using Opamps where the resistors R11

through R22 act as weights. Rpi and Cpi are

the parasitic resistance and capacitance that

present themselves at the input to the Opamp

used in the realization of the i-th neuron.

These parasitic components are included to

model the dynamic nature of the Opamp.

R11

R12

R21

R22

V1

V2

a11

V1+a

12V

2

b1

a21

V1+a

22V

2

b2

x1

u1

x2

u2

Rp1

Cp1

Rp2

Cp2

A

B

Fig. 1 The proposed Neural Network based circuit to solve a system of simultaneous linear equations in 2 variables.

Non-Linear Synapses Weighted Neurons



Non-linear synapses are realized using

comparators. The comparator outputs can be

modelled by equations (1) and (2) given

below.

𝑥1 = 𝑉𝑚 𝑡𝑎𝑛ℎ 𝛽 𝑎11𝑉1 + 𝑎12𝑉2 − 𝑏1 (1)

𝑥2 = 𝑉𝑚 𝑡𝑎𝑛ℎ 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 − 𝑏2) (2)

Where β is the open-loop gain of the

comparator (practically very high) and Vm is

the biasing voltage of the comparator.

Writing node equation for the node ‘A’ in

fig. 1, we get

𝐶1𝑑𝑢1

𝑑𝑡=

𝑥1

𝑅11+

𝑥12

𝑅21− 𝑢1

1

𝑅11+

1

𝑅21+

1

𝑟1

...(3)

Using equation (1) in (3), we get

𝐶1𝑑𝑢1

𝑑𝑡=

𝑉𝑚 tanh 𝛽(𝑎11𝑉1+ 𝑎12𝑉2 − 𝑏1)

𝑅11+


𝑅21−

𝑢1

𝑅1

...(4)

where 𝑅1 = 𝑅11 𝑅21 𝑟1 ...(5)

And similarly

𝐶2𝑑𝑢2

𝑑𝑡=


𝑅12+


𝑅22−

𝑢2

𝑅2

...(6)

where 𝑅2 = 𝑅12 𝑅22 𝑟2 ...(7)

Moreover, it can be shown that the Energy

Function ‘E’ of the network of fig. 1 is

related to the two voltages V1 and V2 by

𝜕𝐸

𝜕𝑉1 = 𝑉𝑚 𝑎11tanh 𝛽(𝑎11𝑉1 + 𝑎12𝑉2 − 𝑏1) +

𝑉𝑚 𝑎21tanh 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 − 𝑏2) ...(8)

𝜕𝐸

𝜕𝑉2 = 𝑉𝑚 𝑎12tanh 𝛽(𝑎11𝑉1 + 𝑎12𝑉2 − 𝑏1) +

𝑉𝑚 𝑎22tanh 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 − 𝑏2) ...(9)

Also, if ‘E’ is the Energy function, it must

satisfy the following conditions [9].

𝜕𝐸

𝜕𝑉1 = 𝐾 𝐶1

𝑑𝑢1

𝑑𝑡 ...(10)

and

𝜕𝐸

𝜕𝑉2 = 𝐾 𝐶2

𝑑𝑢2

𝑑𝑡 ...(11)

where ‘K’ is a constant of proportionality

and has the dimensions of resistance.

Comparing equations (8) & (10), and (9) &

(11) we get

𝑎11 = 𝐾

𝑅11

𝑎12 = 𝐾

𝑅12

...(12)

𝑎21 = 𝐾

𝑅21

𝑎22 = 𝐾

𝑅22

The values of the resistances of fig. 1 can be

easily calculated by choosing a suitable

value of ‘K’ and then using (12).

Although fig. 1 shows the proposed circuit

as applicable to a system of simultaneous

linear equations in two variables, the

technique can be readily extended to solve

n- simultaneous linear equations in n-

variables.



R1

R2

R3

R4

V1

V2

R8

R9R

5

R6

R7

2V1+V

2

5V

1V

R10

R11

R12

V1

- V2

C

D

Fig. 2 The proposed network to solve a sample system of linear equations in two variables.

III. CIRCUIT IMPLEMENTATION AND

SIMULATION RESULTS

This section deals with the application of the

network proposed in the previous section to

a sample problem. Let the simultaneous

linear equations to be solved be

2𝑉1 + 𝑉2 − 5 = 0

𝑉1 − 𝑉2 − 1 = 0

For the chosen set of equations in two

variables (V1 and V2), the circuit of fig. 1

transforms to the circuit presented in fig. 2

above. As shown, some additional circuitry

is needed to generate the inputs to the non-

linear synapses.

The values of the resistances in the circuit of

fig. 2 can be obtained by making note of the

fact that the required signal voltages at the

points marked ‘C’ and ‘D’ are (2V1+V2) and

(V1-V2) respectively. Solving for R5 through

R12, we get

R6 = R7 = R8 = R9 = R10 = R11 = R12 ...(14)

and

2R5 = R7 ...(15)

Choosing R6 = R7 = R8 = R9 = R10 = R11 =

R12 = 3K, we get R5 = 1.5 K.

The values of resistances acting as the

weights on the neurons are obtained from

(12). For the purpose of simulation, the

value of ‘K’ was chosen to be 3K. This was

done to have as fewer different valued

resistances as possible. Using K = 3K in

(12) gives

R12 = R21 = R22 = 3K , and

R11 = 1.5K

The results of SPICE simulation for the

circuit of fig. 2 are presented in fig. 3. From

the plots of the voltages at nodes 1 & 2, it

can be seen that V(1) = 2V and V(2) = 1V.

...(13)



Fig. 3 Simulation results for the circuit of fig. 2

Therefore, the solutions to the simultaneous

linear equations of (13) are V1 = 2V and V2

= 1V. Routine mathematical analysis of (13)

is found to yield the same results.

IV. CONCLUSIONS

A novel neural network based architecture is

presented and utilized to solve simultaneous

linear equations of n- variables. The circuit

thus obtained was realized and simulated

using SPICE for an example set of two

linear equations in two variables. The results

obtained as a result of simulation are found

to verify the theory and are in complete

agreement with the exact mathematical

solution.

REFERENCES

[1] Leonardo V. Ferreira, Eugenius

Kaszkurewicz, and Amit Bhaya, Solving Systems

of Linear Equations Via Gradient Systems With

Discontinuous Righthand Sides: Application to

LS-SVM, IEEE Transactions on Neural

Networks, vol. 16, no. 2, Mar 2005.

[2] Vincent C. Wilburn, Hak-Lim Ko and

Winser E. Alexander, An Algorithm and

Architecture for the Parallel Solution of Systems

of Linear Equations, Fifteenth IEEE Annual

International Phoenix Conference on Computers

and Communications, 1996.

[3] Kechen Zhang, Giorgio Ganis and Martin I.

Sereno, Anti-Hebbian synapses as a linear

equation solver, 1997.

[4] Y. Zhang, D. Jiang and J. Wang, A recurrent

neural network far solving sylvester equation

with time-varying coefficients, IEEE

Transactions on Neural Networks, vol. 13, no.

5, 2002.

[5] C. A. Mead, Analog VLSI and Neural

Systems, Addison-Wesley, United States of

America, 1989.

[6] A. Cichocki and R. Unbehauen, Neural

network for solving systems of linear equations

and related problems, IEEE Transactions on

Circuits and Systems, vol. 39, 1992.

[7] H.T. Kung, Why Systolic Architectures? ,

Computer, vol. 15, January 1982.

[8] S.T. Alexandre, W.E. Alexander, and D.S.

Reeves, An architecture and a simulatior for

parallel signal processing, IEEE Southeastcon

'93 Proceedings, 1993 [9] S. A. Rahman, A Non-linear Synapse Neural

Network and its Applications, Ph. D. Thesis,

2007

Mohd. Samar Ansari received the B.Tech. degree in

Electronics engineering from the Aligarh Muslim

University, Aligarh, India, in 2001 and the M. Tech.

degree in Electronics engineering, with specialization

in Electronic Circuit and System Design, in 2007. He is



teaches Electronic Devices & Circuits and

Microelectronics. His research interests include microelectronics, linear

integrated circuit applications, neural networks, and semiconductor physics.

Syed Atiqur Rahman did B.Sc.(Engg.) in Electrical Engineering in 1988,

and M.Sc.(Engg.) in Electronics and Communication Engineering in 1994,

from Aligarh Muslim University, Aligarh. He obtained his PhD from

Electrical Engineering Department at IIT (Delhi) in 2007. He is currently a

Reader in the Department of Electronics Engineering, Aligarh Muslim

University, where he teaches neural networks and signals & systems. His

fields of interest are Electronic Circuits and Artificial Neural Networks.

Time

0s 0.2ms 0.4ms 0.6ms 0.8ms 1.0ms 1.2ms 1.4ms 1.6ms 1.8ms 2.0ms

V(1) V(2)

0V

1.0V

2.0V

2.5V

978-1-4244-8542-0/10/$26.00 ©2010 IEEE

Employing Differential Voltage Current Conveyor

in Graph Coloring Applications

Mohd. Samar Ansari

Abstract—A non-linear feedback neural network for solving

graph coloring problem is presented. The proposed circuit

employs non-linear feedback, in the form of unipolar

comparators realized using DVCCs and diodes, to introduce

transcendental terms in the energy function ensuring fast

convergence to the solution. PSPICE simulation results on

various random graphs have been presented.

Index Terms— Differential Voltage Current Conveyor

(DVCC), Dynamical Systems, Feedback Neural Networks, Graph

Coloring, Neural Networks, Non – Linear circuits.

I. INTRODUCTION

he graph coloring problem (GCP) has a large number of

applications such as register allocation in digital

computers, frequency channel assignment in mobile

978-1-4244-8542-0/10/$26.00 ©2010 IEEE

x1Z

+

XY2

Y1

DVCC-1

x1

Ix2

Z+

Y2

Y1

DVCC-2

+

_Vi

Rpi Ci

1/gDNi

X

x1

Ixn

Z+

Y2

Y1

DVCC-n

X

V1

V2

Vn

Ix1

Ixn

Ix2

Ix1

Fig. 2 i-th neuron of the proposed non-linear feedback neural circuit to color a

n-node graph.

As can be seen from fig. 2, individual equations from the set

of constraints are passed through non-linear synapses which

are realized using DVCC-based unipolar comparators. The

outputs of the comparators are fed to neurons having weighted

inputs. These weighted neurons are realized by using opamps

where the currents coming from various comparators act as

weights. Weighing is also provided by the self-feedback

resistance (1/gmD). Node equation for node „Ni‟ gives the

equation of motion of the i-th neurons as

∑

[

] (5)

Where, represent the currents after passing through

diodes as shown in Fig. 2.

Substituting the value of results in

∑ *

( – ) +

[

]

(6)

Moreover, it can be shown that the network in fig. 2 can be

associated with an Energy Function „E‟ of the form

∑

∑ ∑ ( ( ))

∑ ∑

∑

∫

(7)

The last term in (7) is usually neglected for high values of the

open-loop gain of the opamp used to realize the neurons. The

first term on the right hand side of (7) is quadratic which tries

to minimize the number of colors. The second term has got a

negative sign. Therefore, the energy function (E) will be

minimized if second term is maximized. This happens when

the voltages corresponding to connected nodes in a graph are

far away from each other. The first two terms on the right

hand side are balancing each other to color a graph properly.

The third term also contributes to lowering of number of

different colors by eliminating all those local minima in the

energy function for which node voltages are negative.

III. SIMULATION RESULTS

The proposed network was tested for various random graphs

using PSPICE simulations. For the opamp, use was made of

the LMC7101A CMOS opamp from National Semiconductor.

The sub-circuit file for this opamp is available in Orcad Model

Library. Similarly, for the diode the model for D1N3063 diode

available in Orcad Library was utilized. The circuit for DVCC

was taken from [7] and is reproduced in figure 3. Standard 0.5

micron CMOS parameters were used for simulation purposes.

The value of for the DVCC was measured to be 1.091 milli-

mhos.

Y1Y2

VBB

M1 M4M2 M3

M5 M6

M9 M10

M7

M11

M8

M12

XZ+

VSS

VDD

M16

Z-

M17 M18

M15M14M13

Fig. 3 CMOS implementation of DVCC [7]

The results of the tests are given in table-I from which it is

seen that the proposed network gives a solution to all the

problems tested and in all the cases the solution is very near to

the chromatic number of the graph. The results of PSPICE

simulations for graph-3 of table-I are presented in figure 4

from which it can be seen that the proposed circuit assigns

only three different voltages to the different nodes.

Table I PSPICE simulation results of the proposed circuit applied

to color different graphs

S. No.

Test Graph

Simulation Results

Chromatic Number No. of

colors

Frequency of

occurrence

1

2 7/10 2

3 3/10

2

2 4/10

2

3 6/10

3

3 10/10 2

978-1-4244-8542-0/10/$26.00 ©2010 IEEE

IV. CONCLUSIONS

In this paper, a CMOS compatible approach to solve graph

coloring problems is presented. The proposed approach

employs n-neurons and n(n-1)-synapses for a n-node graph to

be colored. Each neuron requires one opamp and each synapse

is comprised of DVCCs. This results in significant reduction

in hardware over the existing schemes [x-y]. Specifically the

number of passive floating resistances is limited to one per

neuron. The proposed network was tested on 3 random graphs

and the simulation results confirm the validity of the approach.

V. REFERENCES

[1] T. Zeitlhofer and B. Wess, “A Comparison of Graph Coloring Heuristics

for Register Allocation based on Coalescing in Interval Graphs,” Proc. ISCAS 04, vol. IV, pp. 529 – 532, 2004.

[2] N. A. El-Fishawy, M. M. Hadhood, S. Elnoubi, and W. EL-Sersy, “A

modified Hopfield neural network algorithm for cellular radio channel assignment,” Proc. TENCON 2000, vol. 2, pp. 213 – 216, Sept 2000.

[3] A. D. Blas, A. Jagota and R. Hughey, “Energy Function – Based

Approaches to Graph Coloring,” IEEE Trans. On Neural Networks, vol.

13, no. 1, pp. 81 – 91, Jan 2002.

[4] J. Lui, W. Zhong, and L. Jiao, „Comments on “The 1993 DIMACS

Graph Coloring Challenge” and “Energy Function-Based Approaches to Graph Coloring”,‟ IEEE Trans. On Neural Networks, vol. 17, no. 2, pp.

533, Mar 2006. [5] S. A. Rahman, Jayadeva and S. C. Dutta Roy, “Neural network approach

to graph colouring,” Electronics Letters, vol. 35, no. 14, pp. 1173 –

1175, Jul 1999 [6] C. W. Wu, “Graph Coloring via Synchronization of Coupled

Oscillators,” IEEE Trans. Circuits Syst. I, vol. 45, no. 9, pp. 974 – 978,

Sept 1998. [7] S. Maheshwari, “A canonical Voltage-Controlled VM-APS with a

Grounded Capacitor,” Cir. Sys. Sig. Processing, vol. 27, pages 123-132,

2008.

Fig. 4 PSPICE simulation results for graph-3 in Table-I

Time

0s 10us 20us 30us 40us 50us 60us 70us 80us 90us 100us

V(1) V(2) V(3) V(4) V(5) V(6) V(7) V(8) V(9) V(10)

-5V

0V

5V

10V

15V

International Conference on Intelligent Systems and Networks (ISN-2008), February 22-24, 2008.

Computational Intelligence Laboratory, Institute of Science and Technology, Klawad-133105, Haryana, India

123

Solution of Quadratic Programming Problem

using Feedback Neural Network

M. S. Ansari1 and S. A. Rahman

2

1,2 Department of Electronics Engineering, Z. H. College of Engineering & Technology,

Aligarh Muslim University, Aligarh-202002, India. 1Tel: +91-9927014450, E-mail: [email protected]

2Tel: +91-571-3204389, E-mail: [email protected]

Abstract—A large number of optimization problems, having

non-linear objective functions, are usually approximated by

a second order form. Solving a quadratic programming

problem (QPP) efficiently is one of the most important tasks

in optimization techniques. This is because the solutions of a

QPP using traditional methods include the use of parallel

computing architectures and complex software algorithms.

The time taken by the existing methods to solve a

constrained optimization problem is not always short and

increases significantly with the number of variables. This

paper presents a feedback neural network based hardware

solution to the problem of minimizing a second-order

objective function subject to linear constraints. A direct

consequence of the use of neural networks is massively

parallel processing and fast convergence. This results in a

faster and more compact hardware. Simulation results are

presented for a chosen problem and are found to agree

exactly with the algebraic solution.

I. INTRODUCTION

In the discipline of constrained optimization,

linear programming has been studied the most because of

its fundamental nature and wide applicability. However,

another class of constrained optimization problem is the

quadratic programming problem (QPP). It deals with the

optimization (minimization or maximization) of a second-

order objective function subject to certain linear

constraints. Optimization problems with higher order,

non-linear objective functions are usually approximated

by a second-order form and solved approximately by

standard quadratic programming techniques. Traditional

techniques to solve a QPP normally involve an iterative

process. Due to this iterative nature of the solution

process, several computer algorithms were proposed.

However, the problems to be solved often are of very

large size, so that large computer resources or massive

parallel computer systems with distributed memory are

needed to solve them. When the problem at hand is large,

we must obtain its solution within a reasonably short

period of time. Existing methods, for solving a quadratic

programming problem, include Newton-Barrier method

and sequential quadratic programming (SQP). For all

these methods, the time taken to solve a QPP increases

significantly with the number of variables.

This paper presents a hardware solution to the

problem of solving a quadratic programming problem.

The approach used is to employ neural networks on the

basis of an analog circuit [1-6]. The most important

advantages of neural networks are massively parallel

processing and fast convergence. Hence, architectures

based on neural networks have many computational

advantages over the traditional algorithms running over a

parallel computing system.

The following section lays the basic problem and

the mathematical formulation on which the development

of the proposed network will be based. Section III

contains the circuit implementation of the proposed

network for a set of sample problem in two variables.

SPICE simulation results of the proposed circuit are also

presented. Some concluding remarks are presented in

Section IV.

II. PROPOSED NEURAL NETWORK BASED CIRCUIT

Let the objective function to be minimized be

... (1)

subject to the constraints

where V1 and V2 are the two variables and c11, c12, c22, a11,

a12, a21, a22, b1 and b2 are constants.

The proposed neural-network based circuit to

solve the quadratic programming problem given above is

presented in fig. 1. As can be seen from fig. 1, individual

inequalities from the set of constraints are passed through

a non-linear synapse. Each non-linear synapse is in effect

a unipolar comparator, which is realized using an opamp

and a diode as shown. The outputs of the non-linear

synapses are fed to weighted neurons. These weighted

neurons are again realized by using Opamps where the

resistors R11 through R22, and Rc11 through Rc22, act as

weights. Rpi and Cpi are the parasitic resistance and

capacitance that are present at the input of the opamp used

in the realization of the i-th neuron. These parasitic

components are included to model the dynamic nature of

the Opamp.

Non-linear synapses are realized using

comparators. The comparator outputs can be modelled by

equations (4) and (5) given below.

(2)

(3)



124

..(4)

..(5)

where β is the open-loop gain of the comparator

(practically very high) and Vm is the biasing voltage of the

comparator.

Writing node equation for the node „A‟ in fig. 1, we get

...(6)

Where

..(7)

Similarly, for node „B‟ in figure – 1, we get

...(8)

where

..(9)

Moreover, it can be shown that the Energy

Function „E‟ of the network of fig. 1 is related to the two

voltages V1 and V2 by

...(10)

(11)

Also, if „E‟ is the Energy function, it must satisfy the

following conditions [1, 2, 6]

...(12) And

...(13)

where „K‟ is a constant of proportionality and

has the dimensions of resistance.

Comparing equations (10) & (12), we get

...(14)

A similar comparison between (11) & (13) yields

...(15)

The values of the resistances of fig. 1 can be

easily calculated by choosing a suitable value of „K‟ and

then using (14, 15).

Although fig. 1 shows the proposed circuit as

applicable to a second-order constrained minimization

problem in two variables, the technique can be readily

extended to solve similar problems for higher number of

variables.

III. CIRCUIT IMPLEMENTATION AND SIMULATION

RESULTS

This section deals with the application of the network

proposed in the previous section to a sample problem. Let

the objective function to be minimized be

...(16)



125

subject to

...(17)

The values of resistances acting as the weights

on the neurons are obtained from (14, 15). For the purpose

of simulation, the value of „K‟ was chosen to be 10 KΩ.

Using K = 10 K in (12) gives

Rc11 = Rc11 = Rc11 = Rc11 = K = 10 KΩ

R11 = 1.66 KΩ

R22 = 1 KΩ

R12 = R21 = 2.5 KΩ

Fig. 2 Simulation results for the circuit of fig. 1 applied to minimize

(16,17)

The results of SPICE simulation are presented in fig. 2.

From the plots of the neuron output voltages, it can be

seen that V(1) = -0.58 V and V(2) = 0.41 V. Routine

mathematical analysis of (16) is found to yield the same

results.

IV. CONCLUSIONS

A new neural network based architecture is

presented and utilized to solve a quadratic optimization

problem subject to linear constraints. The circuit thus

obtained was realized and simulated using SPICE for a

sample minimization problem in two variables. The

results obtained from simulation are found to verify the

theory and are in complete agreement with the exact

mathematical solution.

REFERENCES

[1] S. A. Rahman and Mohd. S. Ansari, “Solution of imultaneous Linear Equations using Feedback Neural Network,” Proc.

RASIET-07, International Conference on Recent Applications of

Soft Computing in Engineering and Technology, IET, Alwar, India.

[2] S. J. Arif, S. A. Rahman and Mohd. S. Ansari, “A Non – Linear

Feedback Neural Network for Graph Coloring,” Proc. RASIET-07, International Conference on Recent Applications of Soft

Computing in Engineering and Technology, IET, Alwar, India.

[3] F.L. Filippelli, M. Forti and S. Manetti, “New linear and quadratic programming neural network,” Electronics Letters, vol. 30, no. 20,

September 1994.

[4] M. P. Kennedy, and L. O. Chua, "Neural networks for nonlinear programming," IEEE Trans. 1988, CAS-35, pp. 554-562.

[5] C. Y. Maa, and M. A. Shanblatt, "Linear and quadratic

programming neural network analysis," IEEE Trans., 1992. NN-4, pp. 580-594.

[6] S. A. Rahman, Jayadeva and S. C. Dutta Roy, “Neural network approach to graph colouring,” Electronics Letters, vol. 35, Jul

1999

Fig. 1 The proposed Neural Network based circuit to solve a quadratic programming problem in 2 variables

IMPACT-2009

A non-linear neural circuit for solving system of

simultaneous linear equations

Mohd. Samar Ansari1 and Syed Atiqur Rahman

2

1,2Department of Electronics Engineering, Aligarh Muslim University, Aligarh, India

1Email:[email protected],

2Email:[email protected]

Abstract— A feedback neural network based CMOS

compatible circuit to solve a system of simultaneous linear

equations is presented. The circuit has an associated

transcendental energy function that ensures fast convergence to

the exact solution while enjoying reduction in hardware

complexity over existing schemes. HSPICE simulation results

are presented for a chosen set of equations and are found to

agree exactly with the algebraic solution.

Index Terms—Neural network applications, Neural network

hardware, Nonlinear circuits, linear algebra, linear equations

I. INTRODUCTION

HE solution of linear equations has been a primary goal

for computation since the time of the abacus. References

to word problems requiring the solution of a system of linear

equations appear in ancient Babylonian texts dating back to

circa 300 BC [1]. Some examples of applications requiring

the solution of linear equations are curve fitting, electrical

circuit analysis, multiple correlation as well as real time

applications like real-time speech coding, image processing,

stochastic modeling, and computer-aided realistic three-

dimensional image synthesis [1-9]. Babbage‟s Analytical

Engine (1836) was the first attempt at automating the

equation solving process. It was followed by the Atanasoff-

Berry Computer (1941) which marked the transition from

mechanical to an electronic computing architecture [1, 13].

However, consuming almost a minute for each

addition/multiplication, the Atanasoff-Berry Computer was

certainly slow. The era of fast equation solving computers

was signaled by FPS-164/MAX (1984). It was followed by

the ClearSpeed CSX600 (2005) which was capable of

working at 25 Gflops/s consuming 10 Watts [1]. However,

such architectures as the ClearSpeed CSX600 or the FPS-

164/MAX employ massively parallel digital hardware

employing several coprocessors and running a full blown

operating system. Such an arrangement may not be suitable

for real-time and/or portable applications where a dedicated,

compact, low-power solution is desirable. This has led to

research efforts being directed towards the development of

specialized hardware for the solution of linear equations. The

dynamical system approach was first used by Pyne to solve a

system of linear equations [2]. Later, neural networks

promising massively parallel processing and fast convergence

were also applied to solve linear equations [3, 4, 5].

In this paper, a new non-linear feedback neural architecture

with reduced circuit complexity an energy function involving

transcendental terms which is fundamentally different from

the standard quadratic form associated with Hopfield network

and its variations. Further, from the point of view of

implementation in VLSI technology, the proposed circuit

uses a multi-output transconductance cell in conjunction with

an operational amplifier to emulate a weighted neuron unlike

its predecessor where resistances were utilized to realize the

weights to the neurons [9].

This paper is organized as follows. A brief review of

existing methods for solving linear equations appears in

section – II. Section – III contains the details of the proposed

network along with the design equations. Proofs of the energy

function and validity of the solution are given in section – IV.

Section – V presents the results of HSPICE simulation of the

circuit applied to solve a sample equation set. A discussion

on VLSI implementability of the proposed circuit appears in

section – V. Some conclusive remarks appear in section – VI.

II. EXISTING METHODS

Existing methods to solve a set of linear equations

manually include Gaussian elimination and the Cholesky

decomposition [9, 12]. To determine approximate solutions to

general systems numerically using a software program

running on a computer, the n-dimensional Newton's method

may be used [12]. Software linear equation solving programs

like „SLAE Solver‟ and „Sogol‟ are also commercially

available [12].However, all these methods, being sequential

in nature, are not suitable for real-time processing.

Hardware solutions based on neural networks have been

put forward by Xia, Wang & Hung [5], Cichocki &

Unbehauen [6] and Wang [7]. To solve an n-variable system

of equations, the network of [7] uses three operational

amplifiers, one capacitor and (n+5) resistances to emulate a

single neuron and the time to arrive at the solution is of the

order of hundreds of milliseconds. The network of [6]

provides a significantly improved solution time of around a

microsecond but at the cost of increased hardware

complexity. Each neuron in [6] comprises of three weighted

summers and an inverting integrator. The architecture of [5],

which is a generalized neural network based implementation

of Censor and Elfving’s method for linear inequalities, also

uses an approach similar to [7] utilizing weighted adders and

integrators to realize the neurons.

T

978-1-4244-3604-0/09/$25.00 ©2009 IEEE 120

IMPACT-2009

Ni

Among the current state of the art solutions, the recently

developed concurrent multiprocessor based architectures for

solving linear equations; only systolic/wavefront arrays and

the Block Data Parallel Architecture (BDPA) are suited for

solving computationally intensive problems [9]. However,

even with the extensive simulation tools developed for the

BDPA, there is still a fundamental need to show the

prospective power of such architectures. The ClearSpeed

CSX600 (2005) capable of solution times of the order of

microseconds is among the fastest available linear equation

solver. However, it needs to run a host operating system and

is prone to “OS jitter” at high operating speeds [1]. These

issues together with the fact that its power consumption is

around 10 Watts make the CSX600 unsuitable for real-time

and/or portable applications.

The aim of this paper is to provide a CMOS compatible

neural circuit, the circuit complexity of which compares

favorably with existing networks while enjoying very fast

convergence time.

III. PROPOSED NEURAL NETWORK

Let a system of linear equations in n- variables be

a11 a12 … a1n

a21 a22 … a2n

⋮ ⋮ … ⋮an1 an2 … ann

𝑉1

𝑉2

⋮𝑉𝑛

=

b1

b2

⋮bn

(1)

Where V1, V2,..., Vn are the variables and aij and bi are

constants. The proposed neural-network based circuit to solve

the system of equations of (1) is presented in fig. 1.

As can be seen from fig. 1, individual equations from the

set of equations to be solved are passed through non-linear

synapses which are realized using comparators. The output of

the i-th voltage-mode comparator can be modelled by

𝑥𝑖 = 𝑉𝑚 𝑡𝑎𝑛ℎ 𝛽 𝑎𝑖1𝑉1 + 𝑎𝑖2𝑉2 + ⋯ + 𝑎𝑖𝑛𝑉𝑛 − 𝑏𝑖 (2)

Where β is the open-loop gain of the comparator (practically

very high), ±Vm are the output voltage levels of the

comparator and V1, V2,..., Vn are the neuron outputs.

Vi

Vout+

_

Vi

Vout+

_+

_V

i

b1

b2

x1

x2

Ri

Ci

Vi

Vout+

_

bn x

n

G1

G2

Gn

a11

V1+ a

12V

2 + ...+a

1nV

n

a21

V1+ a

22V

2 +...+ a

2nV

n

an1

V1+ a

n2V

2 + ...+a

nnV

n

g1i

x1

g2i

x2

gni

xn

To other

neurons

Fig. 1 i-th neuron of the proposed feedback neural network circuit to solve

simultaneous linear equations in n-variables

The outputs of the comparators are fed to neurons having

weighted inputs. The neurons are realized by using opamps

and the weights are implemented using transconductance

amplifiers. The currents arriving at the input of a neuron

from various synapses are added using KCL. Rpi and Cpi are

the input resistance and capacitance of the opamp

corresponding to the i-th neuron. These parasitic components

are included to model the dynamic nature of the opamp.

Node equation for node „Ni‟ gives the equation of motion of

the i-th neuron as

𝐶𝑖𝑑𝑢𝑖

𝑑𝑡= 𝑔1𝑖𝑥1 + 𝑔2𝑖𝑥2 + ⋯ + 𝑔𝑛𝑖𝑥𝑛 −

𝑢𝑖

𝑅𝑖 (3)

Where ui is the internal state of the i-th neuron and gji is the

voltage-to-current conversion factor of the j-th transconductance block for the i-th output current. As is shown later in this section, these weights are governed by the entries in the coefficient matrix of (1).

Using (2) in (3) results in

𝐶𝑖𝑑𝑢𝑖

𝑑𝑡= 𝑔1𝑖𝑉𝑚 tanh 𝛽(𝑎11𝑉1 + 𝑎12𝑉2 + ⋯ + 𝑎1𝑛𝑉𝑛 − 𝑏1) +

𝑔2𝑖𝑉𝑚 tanh 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 + ⋯ + 𝑎2𝑛𝑉𝑛 − 𝑏2) + ⋯ +

𝑔𝑛𝑖𝑉𝑚 tanh 𝛽(𝑎𝑛1𝑉1 + 𝑎𝑛2𝑉2 + … + 𝑎𝑛𝑛 𝑉𝑛 − 𝑏𝑛) − 𝑢𝑖

𝑅𝑖 (4)

Moreover, as has been shown in section-IV, the network in

fig. 1 can be associated with an Energy Function „E‟ of the

form

𝐸 =

𝑉𝑚 ln cosh 𝛽 𝑎𝑖𝑗𝑛𝑗=1 𝑉𝑗 − 𝑏𝑖 –

1

𝑅𝑖

𝑛𝑖=1 𝑛

𝑖=1 𝑢𝑖𝑉𝑖

0𝑑𝑉𝑖 (5)


𝜕𝐸

𝜕𝑉1= 𝑉𝑚 𝑎11tanh 𝛽(𝑎11𝑉1 + 𝑎12𝑉2 + ⋯ + 𝑎1𝑛𝑉𝑛 − 𝑏1) +

𝑉𝑚𝑎21tanh 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 + ⋯ + 𝑎2𝑛𝑉𝑛 − 𝑏2) + ⋯ +

𝑉𝑚 𝑎𝑛1tanh 𝛽(𝑎𝑛1𝑉1 + 𝑎𝑛2𝑉2 + ⋯ + 𝑎𝑛𝑛𝑉𝑛 − 𝑏𝑛) − 𝑢1 1

𝑅1 (6)

Also, if „E‟ is the Energy Function, it must satisfy the


𝜕𝐸

𝜕𝑉1= 𝐾 𝐶1

𝑑𝑢1

𝑑𝑡 (7)

Where „K‟ is a constant of proportionality and has the

dimensions of resistance. Comparing (4) and (6) according to

(7) yields

𝑎11 = 𝐾𝑔11 , 𝑎21 = 𝐾𝑔11 , … , 𝑎𝑛1 = 𝐾𝑔11 (8)

The values of the transconductance factors (gji) of fig. 1

can be easily calculated by choosing a suitable value of „K‟

and then using (8). Analysis on similar lines can be

performed to obtain the values of the synaptic weights for the

remaining neurons.

121

IMPACT-2009

IV. ENERGY FUNCTION

Differentiating the energy function of (5) w.r.t. Vi, we get

𝜕𝐸

𝜕𝑉𝑖= 𝑉𝑚𝑎1𝑖 tanh 𝛽(𝑎11𝑉1 + 𝑎12𝑉2 + ⋯ + 𝑎1𝑛𝑉𝑛 − 𝑏1) +

𝑉𝑚𝑎2𝑖 tanh 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 + ⋯ + 𝑎2𝑛𝑉𝑛 − 𝑏2) + … +

𝑉𝑚𝑎𝑛𝑖 tanh 𝛽(𝑎𝑛1𝑉1 + 𝑎𝑛2𝑉2 + … + 𝑎𝑛𝑛 𝑉𝑛 − 𝑏𝑛) − 𝑢𝑖

𝑅𝑖 (9)

Comparing (4) and (6), we observed that

𝜕𝐸

𝜕𝑉𝑖= 𝐾𝐶𝑖

𝑑𝑢𝑖

𝑑𝑡 (10)

Where

𝐾 = 𝑎1𝑖/𝑔1𝑖 ; for all i

Further, the time derivative of the energy function is given by

𝑑𝐸

𝑑𝑡=

𝜕𝐸

𝜕𝑉𝑖

𝑑𝑉𝑖

𝑑𝑡

𝑁𝑖=1 =

𝜕𝐸

𝜕𝑉𝑖

𝑑𝑉𝑖

𝑑𝑢𝑖

𝑁𝑖=1

𝑑𝑢𝑖

𝑑𝑡 (11)


𝑑𝐸

𝑑𝑡= 𝐾𝐶𝑖

𝑁𝑖=1

𝑑𝑢𝑖

𝑑𝑡

2 𝑑𝑉𝑖

𝑑𝑢𝑖 (12)

For the inverting opamp used in the realization of the neuron,

the transfer characteristics are shown in fig. 2. From these

characteristics, it is evident that

𝑑𝑉𝑖

𝑑𝑢𝑖 ≤ 0 (13)

Thereby resulting in

𝑑𝐸

𝑑𝑡≤ 0 (14)

With the equality being valid for

𝑑𝑢𝑖

𝑑𝑡= 0 ; for all i (15)

Equation (14) shows that the energy function can never


energy function. The second criterion viz. the energy function

must have a lower bound is also satisfied for the circuit of

fig.1 wherein it may be seen that V1, V2,..., Vn are all

bounded (being the outputs of opamps) amounting to „E‟

having a definite lower bound.

Fig. 2 Transfer characteristics of the opamp used to realize the neurons

The plot of the energy function for a 2 – variable problem

is shown in fig. 3. From the plot, it is evident that there exists

only one minimum to which the network must converge,

which is the solution of the given system of linear equations.

Fig. 3 Typical energy function plot for a 2 – variable problem

V. SIMULATION RESULTS

The proposed circuit was simulated for solving the

following set of 2 simultaneous linear equations in 2

variables.

2 11 1

V1

V2 =

21.5

(16)

The circuit to solve (16) as obtained from fig. 1 is presented

in fig. 4. As can be seen, some additional circuitry is needed

to generate the inputs to the non-linear synapses. Routine

analysis yields the following values of the resistors at the

input of the non-linear synapses.

R11=2K, R12=4K, R13=4K, R21=3K, R22=3K and R23=3K.

Vi

Vout+

_

Vi

Vout+

_+

+

_

_

V1

V2

b1

b2

x1

x2

Rp1

Rp2

C1

C2

G1

G2

R11

R12 R

13

R21

R22 R

23

a11

x1

a21

x1

a12

x2

a22

x2

Fig. 4 The proposed circuit applied to a 2 – variable problem

For the purpose of HSPICE simulations, the voltage

comparator and the multi-output transconductance amplifier

for each neuron were realized as a single block using a

slightly modified circuit of the differential-input, high-gain,

active-loaded transconductance amplifier [11] shown in fig.

5. Standard BSIM3 0.35µm parameters were used for the

purpose of simulations. The supply voltages were set to VDD

= 15V, VSS = -15V and VBB = -14V. The aspect ratios of the

NMOS and PMOS transistors were taken to be 1.4µm/1.4µm

and 2.8µm/1.4µm respectively. Further, to get the output

currents according to (8) the W/L ratios of transistors M6,

M7, M8 and M9 were set to provide the required current

scaling. Choosing K=1 and using (8) results in Iout1 = 2Iout2 for

G1 block and Iout1 = Iout2 for the G2 block of fig. 4.

Routine mathematical analysis of (16) gives the solution as

V1 = 0.5 and V2 = 1. The results of HSPICE simulation of the

circuit of fig. 4, shown in fig. 6, are found to match exactly

with the algebraic solution.

122

IMPACT-2009

M2

M1

M3

M4

M5

M6

M7

VDD

VSS

VBB

V2

V1 I

out1

M8

M9

Iout2

Fig. 5 CMOS implementation of the comparator and transconductance blocks

Fig. 6 Simulation results for the chosen 2 – variable problem

A qualitative analysis of the simulation results of fig. 6

shows that the voltage-mode outputs obtained are indeed the

correct solutions of (16). The HSPICE output file for the

simulation run gives the value of V1 as 0.499V and V2 as

1.002V which are so near to the algebraic solution that for all

practical purposes they may be considered as exact. Another

noteworthy point in fig. 6 is that the time taken by the circuit

to provide the solutions is significantly reduced as compared

to the time taken by existing schemes [5, 6, 7].

VI. DISCUSSION

This section deals with the monolithic implementation

issues of the proposed circuit. The HSPICE simulations

assumed that all operational amplifiers (and transconductance

amplifiers) are identical, and therefore, analysis is required to

determine how deviations from this assumption affect the

performance of the network. Effects of variations in

component values from one neuron to another also need to be

investigated.

Moreover, for obtaining weighted (scaled) current outputs

use was made of the (W/L) ratios of the transistors M6, M7,

M8 and M9 of fig. 5. This results in a particular circuit

implementation „hard-wired‟ to solve a given set of

equations. Methods to scale the currents by a mechanism

external to the circuit need to be investigated to impart

flexibility to the circuit. Alternative realizations based on the

differential equations (4) governing the system of neurons are

also being considered. Other approaches to obtain the tanh(.)

nonlinearity include the use of a MOSFET operated in the

subthreshold region [11] and the use of Current Differencing

Transconductance Amplifier (CDTA) to provide the same

nonlinearity in the current-mode regime.

VII. CONCLUSIONS

In this paper we have described a CMOS compatible

approach to solve simultaneous linear equations in n

variables, which uses n-neurons and n-synapses. Each neuron

requires one opamp and each synapse is implemented using a

differential transconductance element. This results in

significant reduction in hardware over the existing schemes

[5-7]. The proposed network was tested on a sample problem

set of two simultaneous linear equations and the simulation

results confirm the validity of the approach.

REFERENCES

[1] J. L. Gustafson, “The Quest for Linear Equation Solvers and the

Invention of Electronic Digital Computing,” Proc. JVA‟06, IEEE John

Vincent Atanasoff 2006 International Symposium on Modern Computing, 2006

[2] I. B. Pyne, “Linear programming on an electronic analogue computer,”

Trans. Amer. Inst. Elec. Eng., vol. 75, pp. 139-143, May 1956. [3] K. Zhang, G. Ganis and M. I. Sereno, “Anti-Hebbian synapses as a

linear equation solver,” Proc. International Conference on Neural

Networks, 9 – 12 June 1997, vol. 1, Page(s): 387 – 389

[4] D. Jiang, “Analog Computing for Real-Time solution of Time-Varying Linear Equations,” Proc. ICCCAS, Volume 2, 27-29 June 2004.

[5] Y. Xia, J. Wang and D. L. Hung, “Recurrent Neural Networks for

Solving Linear Inequalities and Equations,” IEEE Tran. Cir. & Sys - I, vol. 46, no. 4, Apr 1999.

[6] A. Cichocki and R. Unbehauen, “Neural Networks for Solving Systems

of Linear Equations and Related Problems,” IEEE Tran. Cir. & Sys - I, vol. 39, no. 2, Feb 1992.

[7] J. Wang, “Electronic Realization of Recurrent Neural Network for

Solving Simultaneous Linear Equations,” Electronics Letters, vol. 28, no. 5, Feb 1992.

[8] M.P. Kennedy, L.O. Chua, “Neural networks for nonlinear

programming,” IEEE Trans. Circuits Sys, vol. 35, no. 5, 1988.

[9] S. A. Rahman and M. S. Ansari, “Solution of Simultaneous Linear

Equations using Feedback Neural Network,” Proc. RASIET, IET,

Alwar, India, Dec 2007.

[10] S. A. Rahman, Jayadeva and S. C. Duttaroy, “Neural network approach

to graph colouring,” Electronics Letters, vol. 35, no. 14, July 1999

[11] R. W. Newcomb and J. D. Lohn, “Analog VLSI for neural networks,”

in The Handbook of Brain Theory and Neural Networks. Cambridge,

MA: The MIT Press, 1998.

[12] Web Link: http://en.wikipedia.org/wiki/Simultaneous_linear_equations

[13] S. White, “A Brief History of Computing - Complete Timeline,” Web

link: http://trillian.randomstuff.org.uk/~stephen//history/timeline.html

123

IMPACT-2009

A Novel Current-mode Non-Linear Feedback Neural Circuit for Solving Linear Equations

Mohd. Samar Ansari1 and Syed Atiqur Rahman2

1,2Department of Electronics Engineering, Aligarh Muslim University, Aligarh, India Emails: [email protected], [email protected]

Abstract— A novel current-mode neural circuit employing non-linear feedback to solve a system of simultaneous linear equations is presented. The circuit has an associated transcendental energy function that ensures fast convergence to the exact solution while enjoying a resistor-less implementation. The hardware complexity of the proposed scheme compares favourably with existing voltage-mode neural circuits for the same task. PSPICE simulation results are presented for a chosen set of equations and are found to be in agreement with the algebraic solution. Index Terms—Neural network applications, Current mode neural network, Neural network hardware, Nonlinear circuits, linear equations, Linear algebra

I. INTRODUCTION

OLVING a system of linear equations Ax = b is one of the most fundamental problems in algebra. Countless applications including, but not limited to, real-time speech coding, image processing, multiple correlation and electrical circuit analysis, occur in the mathematical sciences and engineering. [1-5, 10, 11, 12]. Popular methods to solve large-size problems demand massively parallel computer systems with distributed memory [1, 12]. Apart from such software approaches, some hardware solutions based on artificial neural networks (ANNs) have been proposed over the past decade. Massively parallel processing and fast convergence, which are inherent to neural networks, are utilized to reduce the time taken to arrive at the solution [1-3]. However, some existing neural networks have variable penalty parameters which decrease to zero as time increases to infinity in order to get better accuracy of solution [2]. Therefore, the time taken to arrive at the solution in such cases is not always short [2]. Further, to the best of our knowledge, no current-mode (CM) neural network for solving a system of simultaneous linear equations is reported in the technical literature although the advantages of current-mode (CM) processing in ANNs are well documented [8]. One of the most apparent advantages is that the summing of many signals is most readily accomplished when those signals are currents [8]. Other benefits include better noise immunity and the possibility of resistor-less realizations.

A new current-mode architecture with reduced circuit complexity is presented which avoids the use of penalty parameters. The proposed architecture uses non-linear feedback which leads to a new energy function that involves transcendental terms. This transcendental energy function is fundamentally different from the standard quadratic form associated with Hopfield network and its variants. The proposed circuit uses only multi-output Current Differencing Transconductance Amplifiers (CDTAs) [5] to realize weighted neurons unlike its predecessor where opamps were used to realize neurons and resistances were utilized to realize the weights to the neurons [6]. This paper is organized as follows. A brief review of existing VM methods for solving linear equations appears in section – II. Section – III contains the details of the proposed CM network along with the design equations. Section – IV presents the results of SPICE simulation of the circuit. A discussion on VLSI implementability of the proposed circuit appears in section – V. Some conclusive remarks appear in section – VI.

II. EXISTING VOLTAGE-MODE METHODS Voltage-mode hardware solutions based on neural

networks have been put forward by Wang [2], Cichocki & Unbehauen [3] and Xia, Wang & Hung [4]. To solve an n-variable system of equations, the network of [2] uses three operational amplifiers, one capacitor and (n+5) resistances to emulate a single neuron and the time to arrive at the solution is of the order of hundreds of milliseconds. The network of [3] provides a significantly improved solution time of around a microsecond but at the cost of increased hardware complexity. Each neuron in [3] comprises of three weighted summers and an inverting integrator. The architecture of [4], which is a generalized neural network based implementation of Censor and Elfving’s method for linear inequalities, also uses an approach similar to [2] utilizing weighted adders integrators to realize the neurons.

The aim of this paper is to provide a novel current-mode neural circuit for the solution of a set of simultaneous linear equations using only CDTAs. No passive components are needed for the synaptic interconnections and therefore, the

S

978-1-4244-3604-0/09/$25.00 ©2009 IEEE 284

IMPACT-2009 circuit complexity of the proposed scheme compares favorably with existing networks while enjoying very fast convergence time.

III. PROPOSED NEURAL NETWORK

The schematic symbol of the CDTA is shown in figure 1. The inputs (+) and (-) produce difference current which is transferred to the z terminal and the voltage at the z terminal is converted into output current by a multi-output transconductance stage. The port relations characterizing CDTA are given by

(1)

Fig. 1 Schematic symbol of CDTA

For the purpose of use in the proposed neural circuit, only multi-output X+ type CDTAs are needed. Although the schematic in figure 1 shows only one X+ output, the number of such outputs may readily be increased by repeating the output stages in the actual circuit implementation of the CDTA [5]. Let the system of n- simultaneous linear equations in n- variables be characterized by

(2)

Where I1, I2,..., In are the variables and aij and bi are constants. The proposed neural-network based circuit to solve the system of equations of (2) is presented in figure 2. As can be seen from figure 2, individual equations from the set of equations to be solved are passed through non-linear synapses which are realized using current-mode comparators. The unweighted output of the i-th comparator can be modelled by

(3)

Where β is the open-loop gain of the comparator (practically very high), ±Im are the output current levels of the comparator and I1, I2,..., In are the neuron outputs. Equation (3) shows that the synaptic connections are highly non-linear as opposed to the linear synaptic interconnections used in standard Hopfield network and its variants.

The outputs of the comparators are fed to neurons as weighted inputs. Weights are assigned to the output currents obtained from the comparators according to the entries in the coefficient matrix [aij]. The neurons are realized by using CDTAs wherein the currents arriving at the input of a neuron from various synapses are added. It may be mentioned that the circuit of figure 2 is a current-mode version of the i-th neuron of [6] where opamps were used to implement the non-linear synapses and neurons thereby resulting in voltage-mode operation and generating voltage outputs corresponding to the solutions of the set of linear equations.

Fig. 2 i-th neuron of the proposed current-mode non-linear feedback neural network circuit to solve simultaneous linear equations in n-variables The synaptic weights, or equivalently, the output current scaling factors are equal to the entries in the coefficient matrix of (2). Since the outputs of the comparators are available as currents, there is no need of resistors as were used in [6] to convert the comparator output voltage available into currents that were then added at the input of the amplifier acting as the neuron. Another advantage arising due to current-mode operation is the reduction in the overall power consumed by the circuit to arrive at the answers. This is because the CDTAs used in the network of figure 2 work with currents of the orders of microamperes and can operate from a power supply lower than that normally required for opamps. Moreover, the proposed circuit does not use resistances (floating or grounded) thereby yielding a ‘transistor-only’ implementation. This causes the circuit to compare very favourably with existing circuits from the viewpoint of the suitability of the circuit for monolithic VLSI implementation. Further, since the circuit works in a fashion very similar to its voltage-mode predecessor, the associated energy function will continue to be of the same form as was proposed earlier in [6] and is reproduced below for convenience.

(4)

+_

IDiffIout

+_b1

b2

a1ix1

+_bn

a11I1+ a12I2 + ...+a1nIna21I1+ a22I2 +...+ a2nIn

an1I1+ an2I2 + ...+annIn

To other neuronsa2ix2

anixn

IDiff

IDiff

Iout

Iout

+_

0Iia1iIianiIi Feedback CurrentsCDTA

285

IMPACT-2009

Fig. 3 Bipolar realization of CDTA [5]

IV. RESULTS OF COMPUTER SIMULATION The proposed circuit was simulated for solving the following set of 2 simultaneous linear equations in 2 variables.

(5)

The circuit to solve (5) as obtained from figure 2 is presented in figure 4. As can be seen from figure 2, only one neuron is needed per equation. The neurons are in turn realized by employing only two multi-output CDTAs per neuron.

Fig. 4 The proposed circuit applied to a 2 – variable problem (5)

The circuit of figure 4 was simulated using PSPICE simulation tool. CDTAs were realized using the bipolar junction transistor implementation [5] shown in figure 3. Not explicitly shown in figure 4 is the output port ‘Z’ of the CDTAs. A fixed valued resistor (RZ = 10KΩ) was connected between the Z-output and ground for each of the CDTAs to set the value of the current gain at a high value. The resultant transfer characteristics of the CDTA are presented in figure 5

from which it can be readily verified that the tanh(.) function is indeed obtained as mentioned in (3). For the purpose of PSPICE simulations, the supply voltages were set to VCC = 3V, VEE = -3V and all the biasing currents of the CDTA were kept at 1000µA. Further, to get the output currents according to (5) the emitter areas of transistors in the output stage of the CDTAs were set to provide the required current scaling. The corresponding scaling factors are denoted on the CDTA symbol as ‘X’, and ‘2X’ where ‘2X’ means a current output double that of the ‘X’ output. Routine mathematical analysis of (5) gives the solution as I1 = 15 and I2 = 10. The results of PSPICE simulation of the circuit of fig. 2, shown in figure 6, are found to be I1 = 14.817 µA and I2 = 9.872 µA which are in close agreement with the algebraic solution. Further, another noteworthy point is that the solution time of the proposed circuit is of the order of nanoseconds thereby making it suitable for real-time signal processing applications.

Fig. 5 Transfer characteristics of the CDTA based current-mode comparator

286

IMPACT-2009

Fig. 6 Simulation results for the chosen 2 – variable problem (5)

V. CONCLUSIONS

In this paper we have presented a novel current-mode approach to solve simultaneous linear equations in n variables, which uses n-neurons and n-synapses. Each neuron requires one current-mode amplifier and each synapse is implemented using one current-mode comparator. Both the amplifier and the comparator were implemented using CDTAs. No resistances were needed to realize the weighted synaptic interconnections. This resulted in significant reduction in hardware over the existing schemes [2, 3, and 4].

The proposed network was tested on a sample problem set of 2 simultaneous linear equations and the simulation results confirm the validity of the approach. The time to arrive at the solution was of the order of nanoseconds.

REFERENCES [1] D. Jiang, “Analog Computing for Real-Time solution of Time-Varying

Linear Equations,” Proc. ICCCAS, Volume 2, 27-29 June 2004. [2] Y. Xia, J. Wang and D. L. Hung, “Recurrent Neural Networks for

Solving Linear Inequalities and Equations,” IEEE Tran. Cir. & Sys - I, vol. 46, no. 4, Apr 1999.

[3] A. Cichocki and R. Unbehauen, “Neural Networks for Solving Systems of Linear Equations and Related Problems,” IEEE Tran. Cir. & Sys - I, vol. 39, no. 2, Feb 1992.

[4] J. Wang, “Electronic Realization of Recurrent Neural Network for Solving Simultaneous Linear Equations,” Electronics Letters, vol. 28, no. 5, Feb 1992.

[5] N. A. Shah, M. Quadri and S. Z. Iqbal, “CDTA based universal transadmittance filter,” Analog Integr Circ Sig Process, vol. 52, Page(s): 65–69, Sept. 2007.

[6] S. A. Rahman and M. S. Ansari, “Solution of Simultaneous Linear Equations using Feedback Neural Network,” Proc. RASIET, IET, Alwar, India, Dec 2007.

[7] S. A. Rahman, Jayadeva and S. C. Duttaroy, “Neural network approach to graph colouring,” Electronics Letters, vol. 35, no. 14, July 1999

[8] L. Song, M. I. Elmasry and A. Vannelli, “Analog Neural Network Building Blocks Based on Current Mode Subthreshold Operation,” Proc. ISCAS’93, Page(s):2462 - 2465, 3-6 May 1993.

[9] R. W. Newcomb and J. D. Lohn, “Analog VLSI for neural networks,” in The Handbook of Brain Theory and Neural Networks. Cambridge, MA: The MIT Press, 1998.

[10] Web Link: http://en.wikipedia.org/wiki/Simultaneous_linear_equations [11] J. L. Gustafson, “The Quest for Linear Equation Solvers and the

Invention of Electronic Digital Computing,” Proc. JVA’06, IEEE John Vincent Atanasoff 2006 International Symposium on Modern Computing, 2006

[12] K. Zhang, G. Ganis and M. I. Sereno, “Anti-Hebbian synapses as a linear equation solver,” Proc. International Conference on Neural Networks, 9 – 12 June 1997, vol. 1, Page(s): 387 - 389

287

International MultiConference on Intelligent Systems and Nanotechnology (IISN-2010), February 26-28, 2010 Computational Intelligence Laboratory, Institute of Science and Technology, Klawad-133105, Haryana, India

QPP: Solution using feedback neural network

employing transconducting synaptic interconnections

Mohd. Samar Ansari and Syed Atiqur Rahman

Department of Electronics Engineering, Aligarh Muslim University, Aligarh, India

(Emails: [email protected], [email protected])

Abstract—This paper presents a neural circuit for solving

quadratic programming problems. The objective is to minimize

a quadratic cost function subject to linear constraints. The

proposed circuit employs non-linear feedback, in the form of

unipolar comparators, to introduce transcendental terms in the

energy function ensuring fast convergence to the solution.

Further, the use of resistors to generate weighted inputs to the

neurons is avoided. Instead, transconducting elements are

utilized to directly generate the required scaled currents.

PSPICE simulation results are presented for a chosen

optimization problem and are found to agree with the algebraic

solution.


hardware, Nonlinear circuits, Quadratic Programming,

Optimisation

I. INTRODUCTION

n the discipline of constrained optimization, problems with

nonlinear objective functions are usually approximated by

a second-order (quadratic) system and solved approximately

by a standard quadratic programming technique. Traditional

methods for solving quadratic programming problems

typically involve an iterative process, but long computational

time limits their usage. There is an alternative approach to

solution of this problem. It is to exploit the artificial neural

networks (ANN's) which can be considered as an analog


ANN's have been applied to several classes of constrained




some of the optimization problems. Tank and Hopfield

first proposed a neural network for solving mathematical

programming problems, where a linear programming

problem was mapped into a closed-loop network [2]. Later,

the dynamical approach was extended for solving quadratic



programming problems have been proposed by Kennedy and

Chua [3], Maa and Shanblatt [4], Chen and Feng [5], Wu et

al. [6], Xia [7]. These networks depend on the network

parameters or use expensive analog multipliers. In this paper,

we present a hardware solution to the problem of solving a

quadratic programming problem subject to linear constraints.

The paper is organized as follows. Section II describes the

proposed neural network to minimize a quadratic polynomial

in 2 variables subject to 2 linear constraints. Section III

contains results of PSPICE simulation. Some concluding

remarks appear in section IV.

II. PROPOSED NETWORK


𝐹 =1

2𝐶11𝑉1

2 + 𝐶12𝑉1𝑉2 +1

2𝐶22𝑉2

2 (1)

Subject to

𝑎11𝑉1 + 𝑎12𝑉2 ≤ 𝑏1 (2)

𝑎21𝑉1 + 𝑎22𝑉2 ≤ 𝑏2 (3)

Where V1, and V2 are the variables and aij, bi and cij are

constants. The proposed neural-network based circuit to

minimize (1) subject to (2, 3) is presented in fig. 1.

Vi

Vout

+

_

Vi

Vout

+

_

b1

b2

x1

x2

G1

G2

a11V1+ a12V2

a21V1+ a22V2

gc11 x1

gc12 x2

R11

R21

+

_ V1

Rp1 C1

+_

V2

RpiC2

R22

R12

gc22 x2

gc21 x1

A

B

Fig. 1 The proposed neural network for solving a quadratic programming problem in 2 variables subject to 2 linear constraints

As can be seen from fig. 1, individual equations from the set


are realized using voltage-mode unipolar comparators

followed by multi-output transconducting cells. From fig. 1,

the unipolar comparator outputs can be modeled as

𝑥1 =𝑉𝑚

2 𝑡𝑎𝑛ℎ 𝛽 𝑎11𝑉1 + 𝑎12𝑉2 + 1 (4)

𝑥2 =𝑉𝑚

2 𝑡𝑎𝑛ℎ 𝛽 𝑎21𝑉1 + 𝑎22𝑉2 + 1 (5)

where 𝛽 is the open-loop gain of the comparator (practically

very high), ±Vm are the output voltage levels of the

I


comparator and V1 and V2 are the neuron outputs. The outputs

of the comparators are fed to neurons having weighted inputs.

The neurons are realized by using opamps and the weights

are implemented using transconductance amplifiers. The

currents arriving to the neuron from various synapses get

added up at the input of the neuron. Rpi and Cpi are the input

resistance and capacitance of the opamp corresponding to the

i-th neuron. These parasitic components are included to

model the dynamic nature of the opamp.

Node equation for node ‘A’ gives the equation of motion of

the first neuron in the state space as

𝐶1𝑑𝑢1

𝑑𝑡= 𝑥1𝑔𝑐11 + 𝑥2𝑔𝑐21 +

𝑉1

𝑅11+

𝑉2

𝑅21 −

𝑢1 𝑔𝑐11 + 𝑔𝑐21 +1

𝑅11+

1

𝑅21+

1

𝑅𝑝1 (6)

On simplification, (3) yields

𝐶1𝑑𝑢1

𝑑𝑡= 𝑥1𝑔𝑐11 + 𝑥2𝑔𝑐21 +

𝑉1

𝑅11+

𝑉2

𝑅21 − 𝑢1

1

𝑅1

(7)

Where

1

𝑅1=

1

𝑅11+

1

𝑅21+ 𝑔𝑐11 + 𝑔𝑐21 +

1

𝑅𝑝1 (8)

Similarly, for node ‘B’ we can write

𝐶2𝑑𝑢2

𝑑𝑡= 𝑥1𝑔𝑐12 + 𝑥2𝑔𝑐22 +

𝑉1

𝑅12+

𝑉2

𝑅22 − 𝑢2

1

𝑅2

(9)

Where

1

𝑅2=

1

𝑅12+

1

𝑅22+ 𝑔𝑐12 + 𝑔𝑐22 +

1

𝑅𝑝2 (10)

Where ui is the internal state of the i-th neuron and gcji is the

voltage-to-current conversion factor of the j-th

transconductance block for the i-th output current. As is

shown later in this section, these weights are governed by the

coefficients of (2, 3). Moreover, it can be shown that the

network of fig. 2 may be associated with an ‘energy’ function

of the form

𝐸 =1

2 𝐶11𝑉1

2 + 2𝐶12𝑉1𝑉2 + 𝐶22𝑉22

+𝑉𝑚

𝛽 ln cosh 𝛽 𝑎11𝑉1 + 𝑎12𝑉2– 𝑏1

+𝑉𝑚

𝛽 ln cosh 𝛽 𝑎21𝑉1 + 𝑎22𝑉2– 𝑏2

+𝑉𝑚

2 𝑎11𝑉1 + 𝑎12𝑉2 + 𝑎21𝑉1 + 𝑎22𝑉2 (11)


𝜕𝐸

𝜕𝑉1= 𝑉𝑚 𝑎11 tanh 𝛽(𝑎11𝑉1 + 𝑎12𝑉2 − 𝑏1) +

𝑉𝑚𝑎21tanh 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 − 𝑏2) − 𝑢1 1

𝑅1 (12)



𝜕𝐸

𝜕𝑉1= 𝐾 𝐶1

𝑑𝑢1

𝑑𝑡 (13)

Where ‘K’ is a constant of proportionality and has the

dimensions of resistance. Comparing (7) and (12) according

to (13) using (4) and (5) yields

𝑔11 = 𝑎11

𝐾 , 𝑔21 = 𝑎21

𝐾 (14)

𝑅11 = 𝐾 𝑐 11, 𝑅21 = 𝐾 𝑐 21

(15)

The values of the transconductance factors (gji) of Fig. 1

can be easily calculated by choosing a suitable value of ‘K’

and then using (14). Analysis on similar lines can be

performed to obtain the values of the synaptic weights for the

second neuron and are presented in (16) and (17) below.

𝑔12 = 𝑎12

𝐾 , 𝑔22 = 𝑎22

𝐾 (16)

𝑅12 = 𝐾 𝑐 12, 𝑅22 = 𝐾 𝑐 22

(17)


This section deals with the application of the proposed


𝑉12 + 4𝑉1𝑉2 + 5𝑉2

2 (18)

subject to

𝑉1 − 𝑉2 ≤ −1

𝑉1 + 𝑉2 ≤ 1


are obtained from (15). For the purpose of simulation, the

value of ‘K’ was chosen to be 1 KΩ. Using K = 1 KΩ in (14,

15) gives

gc11 = gc12 = gc21 = gc22 = 1/K = 1 R11 = 1.66 KΩ, R21 = 1 KΩ, R12 = 1.66 KΩ, R22 = 1 KΩ

For the purpose of PSPICE simulations, the voltage

comparator and the multi-output transconductance amplifier

for each neuron were realized as a single block using a

slightly modified circuit of the differential-input, high-gain,

active-loaded transconductance amplifier [9] shown in Fig. 2.

Standard BSIM3 0.35µm parameters were used for the

purpose of simulations. The supply voltages were set to VDD

= 15V, VSS = 0V and VBB = 1V. The aspect ratios of the

NMOS and PMOS transistors were taken to be 1.4µm/1.4µm

and 2.8µm/1.4µm respectively. Further, to get the output

currents according to (14, 16) the W/L ratios of transistors

M6, M7, M8 and M9 were set to provide the required current

scaling. Choosing K=1 and using (14) results in Iout1 = Iout2 for

G1 block and Iout1 = Iout2 for the G2 block.

(19)


M2

M1

M3

M4

M5

M6

M7

VDD

VSS

VBB

V2

V1 I

out1

M8

M9

Iout2

Fig. 2 CMOS implementation of the comparator and transconductance

blocks

Fig. 3 Simulation results for the proposed circuit

Routine mathematical analysis of (18, 19) yields: 𝑉1= -0.584,

𝑉2= 0.416. The results of PSPICE simulation are presented in

fig. 3. From the plots of the neuron output voltages, it can be

seen that V(1) = -0.58 V and V(2) = 0.41 V which are very

near to the algebraic solution thereby confirming the validity

of the approach.

IV. CONCLUSION


approach to solve a quadratic programming problem in 2

variables subject to 2 linear constraints, which uses 2-neurons

and 2-synapses. Each neuron requires one opamp and each

synapse is implemented using a differential transconductance

element. This results in significant reduction in hardware

over the existing schemes [3-8]. The proposed network was

tested on a sample problem of minimizing a quadratic

function in 2 variables and the simulation results confirm the

validity of the approach.

REFERENCES

[1] J.J. Hopfield, D.W. Tank, (1985) ‘Neural’ computation of decisions

optimization problems, Biological Cybern., 52, 141-152.

[2] D.W. Tank, J.J. Hopfield, (1986) Simple neural optimization networks: an A/D converter, signal decision network, and linear programming

circuit, IEEE Trans. Circ. Syst. CAS, 33, 533–541.

[3] M.P. Kennedy, L.O. Chua, (1988) Neural networks for nonlinear programming, IEEE Trans. Circ. Syst., 35, 554–562.

[4] C.Y. Maa, M.A. Shanblatt, (1992) A two-phase optimization neural

network, IEEE Trans. Neural Network 3 (6), 580 – 594. [5] Y.-H. Chen, S.-C. Fang, (1998) Solving convex programming problems

with equality constraints by neural networks, Comput. Math. Appl., 36

(7), 41–68. [6] X.Y. Wu, Y.S. Xia, J. Li, W.K. Chen, (1996) A high performance

neural network for solving linear and quadratic programming problems,

IEEE Trans. Neural Networks, 7 (3), 643–651. [7] Y. Xia, (1996) A new neural network for solving linear and quadratic

programming problems, IEEE Trans. Neural Networks 7 (6), 1544–

1547. [8] S. A. Rahman, Jayadeva, S. C. Dutta Roy, (1999) Neural network

approach to graph colouring, Electronics Letters, 35, 1173 – 1175.

[9] R. W. Newcomb, J. D. Lohn, (1998) Analog VLSI for neural networks, The Handbook of Brain Theory and Neural Networks. Cambridge,

MA: The MIT Press.

A Voltage-Mode Analog Circuit for Solving Linear Programming Problems

Mohd. Samar Ansari

Department of Electronics Engineering

Aligarh Muslim University

Aligarh, India email: [email protected]

Syed Atiqur Rahman

Department of Electronics Engineering

Aligarh Muslim University

Aligarh, India email: [email protected]

Abstract—This paper presents a neural circuit for solving linear

programming problems. The objective is to minimize a first order

cost function subject to linear constraints. The dynamic analog

circuit, consisting of N identical units for N variable problem, can

solve the general LPP and always converges to the optimal

solution in constant time, irrespective of the initial conditions,

which is of the order of its time constant. The proposed circuit

employs non-linear feedback, in the form of unipolar comparators,

to introduce transcendental terms in the energy function ensuring

fast convergence to the solution. Further, the use of resistors to

generate weighted inputs to the neurons is avoided. Instead,

transconducting elements are utilized to directly generate the

required scaled currents. PSPICE simulation results are presented

for a chosen optimization problem and are found to agree with the

algebraic solution.


hardware, Nonlinear circuits, Linear Programming,

Optimization

I. INTRODUCTION

inear programming is a considerable field of optimization for

several reasons. Many practical problems in operations

research can be expressed as linear programming problems.

Certain special cases of linear programming, such as network flow

problems and multicommodity flow problems are considered

important enough to have generated much research on specialized

algorithms for their solution [1-6]. A number of algorithms for

other types of optimization problems work by solving LP

problems as sub-problems. In an LPP problem, a linear function is

optimized subject to certain linear constraints. The general LPP

problem can be stated as:

Minimize 𝑷𝑇𝑽 (1)

Subject to 𝒂𝑽 ≤ 𝒃 (2)

where 𝑽RN is a column vector of decision variables, 𝑷RN is a

column vector of cost coefficient, 𝒃RM is a column vector of

constant bounds, 𝒂RMxN is a constraint coefficient matrix, M

N, and the superscript T denotes the transpose operator. The M

constraints specified by (2) define the feasible region in the N

dimensional Euclidean space in which the cost function is required

to be minimized [2].

Traditional methods for solving linear programming problems

typically involve an iterative process, but long computational time

limits their usage. There is an alternative approach to solution of

this problem. It is to exploit the artificial neural networks (ANN's)

which can be considered as an analog computer relying on a

highly simplified model of neurons [1]. ANN's have been applied

to several classes of constrained optimization problems and have

shown promise for solving such problems more effectively. For

example, the Hopfield neural network has proven to be a powerful

tool for solving some of the optimization problems. Tank and

Hopfield first proposed a neural network for solving

mathematical programming problems, where a linear

programming problem was mapped into a closed-loop network

[2]. Over the past two decades several neural-network

architectures for solving linear programming problems have been

proposed by Wang [3], Xia [4], Xia, Wang & Hung [5] and Malek

and Yari [6]. Wang‘s realization of a neural network for solving

linear programming problems utilizes 3n amplifiers, n2+3n

resistances and n capacitors. The circuit of [4] requires 2m2+4mn

amplifiers, 2m2+4mn+3m+3 summers, n+m integrators, and n

simple limiters. The circuit of [5] consists of m2+2mn amplifiers,

3n summers, n integrators, and n simple limiters. And although no

circuit implementation of the proposed scheme is given in [6], it is

evident that it would not be simple to actually realize in hardware.

Therefore, the search still continues for a working circuit, to solve

LPP, which is feasible to implement in integrated forms and is fast

enough for utilization in real-time applications where the time-to-

solve is needed to be of the order of tenths of milliseconds [5].

In this paper, a hardware solution to the problem of solving a

linear programming problem subject to linear inequality

constraints is presented. The proposed circuit employs only n

amplifiers, m comparators and mn+m+n resistances. It is evident

that the circuit complexity of the proposed scheme is much

reduced as compared to existing ones [3-6].


proposed neural network to minimize a first order polynomial in 2

variables subject to 2 linear constraints. Section III contains results

of PSPICE simulation. Some concluding remarks appear in

section IV.



𝐹 = 𝑃1𝑉1 + 𝑃2𝑉2 (3) Subject to

𝑎11𝑉1 + 𝑎12𝑉2 ≤ 𝑏1 (4)

𝑎21𝑉1 + 𝑎22𝑉2 ≤ 𝑏2 (5)

Where V1, and V2 are the variables and aij, bi and cij are constants.

The proposed neural-network based circuit to minimize (3) subject

to (4, 5) is presented in fig. 1.

L

Vi

Vout

+

_

Vi

Vout

+

_

b1

b2

x1

x2

G1

G2

a11V1+ a12V2

a21V1+ a22V2

gc11 x1

gc12 x2

R1

+

_

V1Rp1 C1

+_

V2RpiC2

R2

gc22 x2

gc21 x1

A

B

P1

P2

Fig. 1 The proposed neural network for solving a linear programming

problem in 2 variables subject to 2 linear constraints As can be seen from fig. 1, individual equations from the set of

constraints are passed through non-linear synapses which are

realized using voltage-mode unipolar comparators followed by

multi-output transconducting cells. From fig. 1, the unipolar

comparator outputs can be modeled as

𝑥1 =𝑉𝑚

2 𝑡𝑎𝑛ℎ 𝛽 𝑎11𝑉1 + 𝑎12𝑉2 + 1 (6)

𝑥2 =𝑉𝑚

2 𝑡𝑎𝑛ℎ 𝛽 𝑎21𝑉1 + 𝑎22𝑉2 + 1 (7)

where 𝛽 is the open-loop gain of the comparator (practically very

high), ±Vm are the output voltage levels of the comparator and V1

and V2 are the neuron outputs. The outputs of the comparators are

fed to neurons having weighted inputs. The neurons are realized

by using opamps and the weights are implemented using

transconductance amplifiers. The currents arriving to the neuron

from various synapses get added up at the input of the neuron. Rpi

and Cpi are the input resistance and capacitance of the opamp

corresponding to the i-th neuron. These parasitic components are

included to model the dynamic nature of the opamp.

Node equation for node ‗A‘ gives the equation of motion of the

first neuron in the state space as

𝐶1𝑑𝑢1

𝑑𝑡= 𝑥1𝑔𝑐11 + 𝑥2𝑔𝑐21 +

𝑃1

𝑅1 − 𝑢1 𝑔𝑐11 +

𝑔𝑐21 +1

𝑅1+

1

𝑅𝑝1 (8)

On simplification, (3) yields

𝐶1𝑑𝑢1

𝑑𝑡= 𝑥1𝑔𝑐11 + 𝑥2𝑔𝑐21 +

𝑃1

𝑅1 − 𝑢1

1

𝑅𝑒𝑞 1 (9)

Where

1

𝑅𝑒𝑞 1=

1

𝑅1+ 𝑔𝑐11 + 𝑔𝑐21 +

1

𝑅𝑝1 (10)

Similarly, for node ‗B‘ we can write

𝐶2𝑑𝑢2

𝑑𝑡= 𝑥1𝑔𝑐12 + 𝑥2𝑔𝑐22 +

𝑃2

𝑅2 − 𝑢2

1

𝑅𝑒𝑞 2 (11)

Where

1

𝑅𝑒𝑞 2=

1

𝑅2+ 𝑔𝑐12 + 𝑔𝑐22 +

1

𝑅𝑝2 (12)

Where ui is the internal state of the i-th neuron and gcji is the

voltage-to-current conversion factor of the j-th transconductance

block for the i-th output current. As is shown later in this section,

these weights (gcji) are governed by the coefficients of (4, 5).

Moreover, it can be shown that the network of fig. 1 may be

associated with an ‗energy‘ function of the form

𝐸 = 𝑃1𝑉1 + 𝑃2𝑉2 + 𝑉𝑚

𝛽 ln cosh 𝛽 𝑎11𝑉1 +

𝑎12𝑉2– 𝑏1 +𝑉𝑚

𝛽 ln cosh 𝛽 𝑎21𝑉1 + 𝑎22𝑉2– 𝑏2 +

𝑉𝑚

2 𝑎11𝑉1 + 𝑎12𝑉2 + 𝑎21𝑉1 + 𝑎22𝑉2 (13)


𝜕𝐸

𝜕𝑉1= 𝑃1 + 𝑉𝑚 𝑎11tanh 𝛽(𝑎11𝑉1 + 𝑎12𝑉2 − 𝑏1) +

𝑉𝑚𝑎21tanh 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 − 𝑏2) (14)

Also, if ‗E‘ is the Energy Function, it must satisfy the following

condition [7].

𝜕𝐸

𝜕𝑉1= 𝐾 𝐶1

𝑑𝑢1

𝑑𝑡 (15)

Where ‗K‘ is a constant of proportionality and has the dimensions

of resistance. Comparing (9) and (14) according to (15) using (6)

yields

𝑔𝑐11 = 𝑎11

𝐾 , 𝑔𝑐21 = 𝑎21

𝐾 (16)

𝑅1 = 𝐾 (17)

The values of the transconductance factors (gji) of Fig. 1 can be

easily calculated by choosing a suitable value of ‗K‘ and then

using (16). Analysis on similar lines can be performed to obtain

the values of the synaptic weights for the second neuron and are

presented in (18) and (19) below.

𝑔𝑐12 = 𝑎12

𝐾 , 𝑔𝑐22 = 𝑎22

𝐾 (18)

𝑅2 = 𝐾 (19)


This section deals with the application of the proposed network to

task of minimizing the objective function

5𝑉1 + 2𝑉2 (20) subject to

𝑉1 + 𝑉2 ≥ 1

𝑉2 ≤ 1

The values of resistances acting as the weights on the neurons are

obtained from (17, 19). For the purpose of simulation, the value of

‗K‘ was chosen to be 1 KΩ. Using K = 1 KΩ in (16 through 19)

gives

gc11 = gc21 = gc22 = 1/K = 1; gc12 = 0 R1 = 1 KΩ, R2 = 1 KΩ

For the purpose of PSPICE simulations, the voltage comparator

and the multi-output transconductance amplifier for each neuron

were realized as a single block using a slightly modified circuit of

(21)

(22)

the differential-input, high-gain, active-loaded transconductance

amplifier [8] shown in Fig. 2. Standard BSIM3 0.35µm parameters

were used for the purpose of simulations. The supply voltages

were set to VDD = 15V, VSS = 0V and VBB = 1V. The aspect

ratios of the NMOS and PMOS transistors were taken to be

1.4µm/1.4µm and 2.8µm/1.4µm respectively. Further, to get the

output currents according to (22) the W/L ratios of transistors M6,

M7, M8 and M9 were set to provide the required current scaling.

M2

M1

M3

M4

M5

M6

M7

VDD

VSS

VBB

V2

V1 I

out1

M8

M9

Iout2

Fig. 2 CMOS implementation of the comparator and transconductance

blocks

Routine mathematical analysis of (20, 21) yields: 𝑉1= 0, 𝑉2= 1.

The results of PSPICE simulation are presented in fig. 3. From the

plots of the neuron output voltages, it can be seen that V(1) = 0 V

and V(2) = 1 V which correspond exactly to the algebraic solution

thereby confirming the validity of the approach.


IV. CONCLUSION

In this paper, a CMOS compatible approach to solve a linear

programming problem in 2 variables subject to 2 linear

constraints, which uses 2-neurons and 2-synapses, is described.

Each neuron requires one opamp and each synapse is implemented

using a differential transconductance element. This results in

significant reduction in hardware over the existing schemes [3-6].

The proposed network was tested on a sample problem of

minimizing a quadratic function in 2 variables and the simulation

results confirm the validity of the approach.

REFERENCES

[1] J.J. Hopfield, D.W. Tank, ―‗Neural‘ computation of decisions

optimization problems,‖ Biological Cybern., 52, 141-152, 1985

[2] D. W. Tank and J. J. Hopfield, ―Simple Neural Optimization Networks: An A/D Converter, Signal Decision Circuit, and A Linear

Programming Circuit,” IEEE Trans. Circuits and Systems, Vol.

CAS-33, No.5, pp. 533-541, May 1986. [3] J. Wang, ―Analysis and Design of a Recurrent Neural Network for

Linear Programming‖, IEEE Trans. Circuits and Systems–1:

Fundamental Theory and Applications, Vol.40, No.9, pp. 613-618, Sept.1993.

[4] Y. Xia, ―A New Neural Network for Solving Linear and Quadratic

Programming Problems,‖ IEEE Trans. Neural Networks, Vol.7, No.6, pp. 1544-1547, Nov 1996.

[5] Y. Xia, J. Wang and D. L. Hung, ―Recurrent Neural Networks for

Solving Linear Inequalities and Equations,‖ IEEE Trans. Circuits and Systems–1: Fundamental Theory and Applications, Vol.46,

No.4, pp. 452-462, April 1999.

[6] A. Malek and A. Yari., ―Primal–dual solution for the linear programming problems using neural networks,‖ Applied

Mathematics and Computation, 167, 198–211, 2005.

[7] S. A. Rahman, Jayadeva, S. C. Dutta Roy, ―Neural network approach to graph colouring,‖ Electronics Letters, 35, 1173 – 1175,

1999

[8] M. S. Ansari and S. A. Rahman, ―A non-linear neural circuit for solving system of simultaneous linear equations,‖ Proc. IMPACT-

2009, AMU, Aligarh, March 2009.

CERA09

DVCC-Based Non-Linear Feedback Neural Circuit

for Solving Quadratic Programming Problems


1,2Department of Electronics Engineering, Aligarh Muslim University, Aligarh, India

(Emails:[email protected], [email protected]

2)

Abstract— This paper presents a non-linear feedback neural

network for solving quadratic programming problems. The

objective is to minimize a quadratic cost function subject to

linear constraints. The proposed circuit employs non-linear

feedback, in the form of unipolar comparators realized using

DVCCs and diodes, to introduce transcendental terms in the

energy function ensuring fast convergence to the solution.

PSPICE simulation results are presented for a chosen

optimization problem and are found to agree with the algebraic

solution.

Index Terms— Neural Networks, Non – Linear circuits,

Feedback Neural Networks, Quadratic Programming,

Dynamical Systems

I. INTRODUCTION

uadratic Programming problems are very important in the

field of optimization. They arise in many applications

such as constrained least mean square estimation and the

classical newsvendor problem. Besides its wide applications,

quadratic programming is also of theoretic meaning, because

it forms a basis for solving some general nonlinear

programming problems. In the discipline of constrained

optimization, problems with nonlinear objective functions are

usually approximated by a second-order (quadratic) system

and solved approximately by a standard quadratic

programming technique. Traditional methods for solving

quadratic programming problems typically involve an

iterative process, but long computational time limits their

usage. There is an alternative approach to solution of this

problem. It is to exploit the artificial neural networks

(ANN's) which can be considered as an analog computer

relying on a highly simplified model of neurons [1]. ANN's

have been applied to several classes of constrained




some of the optimization problems. Tank and Hopfield

first proposed a neural network for solving mathematical

programming problems, where a linear programming

problem was mapped into a closed-loop network [1]. Later,

the dynamical approach was extended for solving quadratic



programming problems have been proposed by Kennedy and

Chua [2], Maa and Shanblatt [3], Wu et al. [4], Xia [5].

These networks depend on the network parameters or use

expensive analog multipliers. More recently, Malek and

Alipour [6] proposed a recurrent neural network that is able

to solve quadratic programming problems. This architecture

is advantageous in the sense that there is no need to set

network parameters. Moreover, the number of analog

multipliers is reduced. This reduces the hardware complexity

to a certain extent. However, the network still has a complex

hardware in the sense that the main processing element

(alpha) would be cumbersome to implement in hardware. In

this paper, we present a hardware solution to the problem of

solving a quadratic programming problem subject to linear

constraints. The proposed architecture uses non-linear

feedback which leads to a new energy function that involves

transcendental terms. This transcendental energy function is

fundamentally different from the standard quadratic form

associated with Hopfield network and its variants. The

hardware complexity of the proposed circuit compares

favourably with the existing hardware implementations.

The remainder of this paper is arranged as follows:

Section–II outlines the basic problem and the mathematical

formulation on which the development of the proposed

network will be based. Section–III contains the circuit

implementation of the proposed network for a set of sample

problem in two variables. SPICE simulation results of the

proposed circuit are also presented. Issues that are expected

to arise in actual monolithic implementations are discussed in

Section–IV. Concluding remarks are presented in Section–V.

II. PROPOSED NEURAL NETWORK


𝐹 =1

2𝐶11𝑉1

2 + 𝐶12𝑉1𝑉2 +1

2𝐶22𝑉2

2 (1)

Subject to

𝑎11𝑉1 + 𝑎12𝑉2 ≤ 𝑏1 (2)

𝑎21𝑉1 + 𝑎22𝑉2 ≤ 𝑏2 (3)



minimize (1) subject to (2, 3) is presented in fig. 1.

Q

CERA09

b1

x1s11Ix

Z+

1

XY2

Y1

DVCC-1

a11V1+ a12V2

Z+

2

s12Ix

b2

x1s21Ix

Z+

1

XY2

Y1

DVCC-2

a21V1+ a22V2

Z+

2

s22Ix

+

_V1

+_

Rp2Cp2

V2

Rp1 Cp1

R11

R22

R21

R12

A

B

Fig. 1 The proposed feedback neural circuit to solve a quadratic

programming problem in 2-variables with 2 linear constraints

As can be seen from fig. 1, individual equations from the

set of constraints are passed through non-linear synapses

which are realized using DVCC-based unipolar comparators.

x1

Z+

X

Y2

Y1

DVCC

Z-

VY1

VY2

VX

IX

IZ+

IZ-

Fig. 2 Symbolic Representation of DVCC

The symbolic diagram of the DVCC is shown in fig. 2. The

device can be characterized by the following port relations:

𝑉𝑋

𝐼𝑌1

𝐼𝑌2

𝐼𝑍+

𝐼𝑍−

=

0001

−1

10000

−10000

00000

00000

𝐼𝑋𝑉𝑌1

𝑉𝑌2

𝑉𝑍+

𝑉𝑍−

(4)

Although both Z+ and Z- types of current outputs are

mentioned in (4), the DVCCs used in the proposed network

use only Z+ type of outputs. For the DVCC, comparator

action can be achieved by putting 𝑅𝑋 → 0 i.e. by directly

grounding the X-terminal [7]. For such a case, the current in

the X-port will saturate and can be written as

𝐼𝑋 = 𝐼𝑚 tanh 𝛽 𝑉𝑌1– 𝑉𝑌2 (5)


very high) and ±𝐼𝑚 are the saturated output current levels of

the comparator. Equation (5) can also be written in an

equivalent notation as

𝐼𝑋 = 𝑔𝑉𝑚 tanh 𝛽 𝑉𝑌1– 𝑉𝑌2 (6)

Where 𝑔 is the transconductance from the input voltage ports

(Y1 and Y2) to the output current port (X) and ±𝑉𝑚 are the

biasing voltages of the DVCC-based comparator.

By virtue of DVCC action, this current will be transferred to

the Z+ ports as

𝐼𝑧+ = 𝑠𝐼𝑋 = 𝑠𝑔𝑉𝑚 tanh 𝛽 𝑉𝑌1– 𝑉𝑌2 (7)

Where 𝑠 is the current scaling factor introduced during the

current conveying process from the X-port to the Z+ port.

Ideally, 𝑠 = 1. However, by altering the aspect ratios of the

transistors used in the output stages of the DVCC, 𝐼𝑧+ can be

made a scaled replica of 𝐼𝑋 . In that case, the value of 𝑠 will be

different from unity. This property will be utilized in the

design of the multi-output DVCCs needed for the proposed

circuit.

The outputs of the comparators are fed to neurons having

weighted inputs. These weighted neurons are realized by

using opamps where the scaled currents coming from various

comparators act as weights. Weighing is also provided by the

self-feedback resistances (R11 & R22 for neurons 1 & 2

respectively) and the cross-feedback resistances (R21 and R12

for neurons 1 & 2 respectively). Rpi and Cpi are the input

resistance and capacitance of the opamp corresponding to the

i-th neuron. These parasitic components are included to

model the dynamic nature of the opamp.

Node equation for nodes ‘A’ and ‘B’ give the equations of

motions of the first and second neurons as

𝐶𝑝1𝑑𝑢1

𝑑𝑡= 𝑠11𝐼′𝑋1 + 𝑠21𝐼′𝑋2 +

𝑉1

𝑅11+

𝑉2

𝑅21 − 𝑢1

1

𝑅1 (8)

Where, 𝑠11𝐼′𝑋1 and 𝑠21𝐼′𝑋2 represent the currents 𝑠11𝐼𝑋1 and

𝑠21𝐼𝑋2after passing through diodes as shown in Fig. 1 and

1

𝑅1=

1

𝑅𝑝1+

1

𝑅11+

1

𝑅21 (9)

And,

𝐶𝑝2𝑑𝑢2

𝑑𝑡= 𝑠12𝐼′𝑋1 + 𝑠22𝐼′𝑋2 +

𝑉1

𝑅12+

𝑉2

𝑅22 − 𝑢2

1

𝑅2 (10)

Where, 𝑠12𝐼′𝑋1 and 𝑠22𝐼′𝑋2 represent the currents 𝑠12𝐼𝑋1 and

𝑠22𝐼𝑋2 after passing through diodes as shown in Fig. 1 and

1

𝑅2=

1

𝑅𝑝2+

1

𝑅12+

1

𝑅22 (11)

In equations (8) and (10) ui is the internal state of the i-th

neuron and 𝑠𝑗𝑖 is the current scaling factor at the j-th output of

i-th DVCC. As is shown later in this section, these weights

are governed by the constraint inequalities (2, 3). Using (6) in

(8) and (9) results in (12) and (13) given below.

𝐶𝑝1𝑑𝑢1

𝑑𝑡=

1

2𝑠11𝑔𝑉𝑚 tanh 𝛽 𝑎11𝑉1 + 𝑎12𝑉2– 𝑏1 + 1 +

1

2𝑠21 𝑔𝑉𝑚 tanh 𝛽 𝑎21𝑉1 + 𝑎22𝑉2– 𝑏2 + 1 +

𝑉1

𝑅11+

𝑉2

𝑅21 − 𝑢1

1

𝑅1 (12)

𝐶𝑝2𝑑𝑢2

𝑑𝑡=

1

2𝑠12𝑔𝑉𝑚 tanh 𝛽 𝑎11𝑉1 + 𝑎12𝑉2– 𝑏1 + 1 +

1

2𝑠22 𝑔𝑉𝑚 tanh 𝛽 𝑎21𝑉1 + 𝑎22𝑉2– 𝑏2 + 1 +

𝑉1

𝑅12+

𝑉2

𝑅22 − 𝑢2

1

𝑅2 (13)

CERA09

Moreover, it can be shown that the network in fig. 1 can be

associated with an Energy Function ‘E’ of the form


𝜕𝐸

𝜕𝑉1= 𝑐11𝑉1 + 𝑐21𝑉2 +

1

2𝑉𝑚 𝑎11tanh 𝛽(𝑎11𝑉1 + 𝑎12𝑉2 − 𝑏1) +

1

2𝑉𝑚𝑎21tanh 𝛽(𝑎21𝑉1 + 𝑎22𝑉2 − 𝑏2) − 𝑢1

1

𝑅1 (15)



𝜕𝐸

𝜕𝑉1= 𝐾 𝐶1

𝑑𝑢1

𝑑𝑡 (16)

Where ‘K’ is a constant of proportionality and has the

dimensions of resistance. Comparing (12) and (15) according

to (16) yields

𝑎11 = 𝑔𝐾𝑠11; 𝑎21 = 𝑔𝐾𝑠21 (17)

𝑐11 = 𝐾

𝑅11; 𝑐21 =

𝐾

𝑅21 (18)

The values of the current scaling factors (sji) of Fig. 1 can

be easily calculated by setting the value of ‘K’ to be equal to 1

𝑔 and then using (17). The values of the weight resistances

connected to the first neuron, R11 and R21, can be obtained by

using (18). Analysis on similar lines can be performed to

obtain the values of the synaptic weights for the second

neuron.




3𝑉12 + 4𝑉1𝑉2 + 5𝑉2

2 (19)

subject to

𝑉1 − 𝑉2 ≤ −1

𝑉1 + 𝑉2 ≤ 1


are obtained from (18). Towards that end, first the value of 𝑔

was computed for the DVCC which was found to be 1.091

milli-mhos. The corresponding values of the weight

resistances are found to be: R11 =305.53Ω; R21 =229.14Ω; R12

=229.14Ω; R22 =183.318Ω. The values of the current scaling

factors were found out to be: s11 = 1; s12 = -1; s21 = 1 ; s22 = 1.

The circuit for DVCC was taken from [7] and standard 0.5

micron CMOS parameters were used for simulation purposes.

For the opamp, use was made of the LMC7101A CMOS

opamp from National Semiconductor. The sub-circuit file for

this opamp is available in Orcad Model Library.

Routine mathematical analysis of (19, 20) yields: 𝑉1= -0.584,

𝑉2= 0.416. The results of PSPICE simulation are presented in

fig. 3. From the plots of the neuron output voltages, it can be

seen that V(1) = -0.58 V and V(2) = 0.41 V which are very

near to the algebraic solution thereby confirming the validity

of the approach.

Fig. 3 Simulation results for the chosen 2 – variable problem

IV. CONCLUSIONS


approach to solve a quadratic programming problem in 2

variables subject to 2 linear constraints, which uses 2-neurons

and 2-synapses. Each neuron requires one opamp and each

synapse is implemented using a dual output DVCC. This

results in significant reduction in hardware over the existing

schemes [2-6]. The proposed network was tested on a sample

problem of minimizing a quadratic function in 2 variables

and the simulation results confirm the validity of the

approach.

REFERENCES

[1] D.W. Tank, J.J. Hopfield, (1986) Simple neural optimization networks:

an A/D converter, signal decision network, and linear programming

circuit, IEEE Trans. Circ. Syst. CAS, 33, 533–541. [2] M.P. Kennedy, L.O. Chua, (1988) Neural networks for nonlinear

programming, IEEE Trans. Circ. Syst., 35, 554–562.

[3] C.Y. Maa, M.A. Shanblatt, (1992) A two-phase optimization neural network, IEEE Trans. Neural Network 3 (6), 580 – 594.

[4] X.Y. Wu, Y.S. Xia, J. Li, W.K. Chen, (1996) A high performance

neural network for solving linear and quadratic programming problems, IEEE Trans. Neural Networks, 7 (3), 643–651.

[5] Y. Xia, (1996) A new neural network for solving linear and quadratic

programming problems, IEEE Trans. Neural Networks 7 (6), 1544–1547.

[6] A. Malek, M. Alipour, (2007), Numerical solution for linear and

quadratic programming problems using a recurrent nueral network, Appl. Math. Computation, 192, 27 – 39.

[7] S. Maheshwari, (2008) A canonical voltage-controlled VM-APS with a

grounded capacitor,” Cir. Syst. Sig. Process, 27, 123-132. [8] S. A. Rahman, Jayadeva, S. C. Dutta Roy, (1999) Neural network

approach to graph colouring, Electronics Letters, 35, 1173 – 1175.

(20)

978-1-4244-8542-0/10/$26.00 ©2010 IEEE

A DVCC-based Non-Linear Analog Circuit for

Solving Linear Programming Problems


Abstract—This paper presents a neural circuit for solving

linear programming problem (LPP). The objective is to minimize

a first order cost function subject to linear constraints. The

dynamic analog circuit, consisting of N identical units for N

variable problem, can solve the general LPP and always

converges to the optimal solution in constant time, irrespective of

the initial conditions, which is of the order of its time constant.

The proposed circuit employs non-linear feedback, in the form of

Differential Voltage Current Conveyor (DVCC) based unipolar

comparators, to introduce transcendental terms in the energy

function ensuring fast convergence to the solution. Further, the

use of resistors to generate weighted inputs to the neurons is

avoided. Instead, DVCCs are utilized to directly generate the

required scaled currents. PSPICE simulation results are

presented for a chosen optimization problem and are found to

agree with the algebraic solution.

Index Terms— DVCC, Linear Programming, Neural network

applications, Nonlinear circuits, Optimization.

I. INTRODUCTION

inear programming is a considerable field of optimization

for several reasons. Many practical problems in

operations research can be expressed as linear programming

problems. Certain special cases of linear programming, such

as network flow problems and multi-commodity flow

problems are considered important enough to have generated

much research on specialized algorithms for their solution [1-

6]. A number of algorithms for other types of optimization

problems work by solving LP problems as sub-problems. In a

LPP problem, a linear function is optimized subject to certain

linear constraints. The general LPP problem can be stated as:

Minimize (1)

Subject to (2)

where Rn is a column vector of decision variables, R

n is

a column vector of cost coefficient, Rm is a column vector

of constant bounds, Rm x n

is a constraint coefficient matrix,

m n, and the superscript T denotes the transpose operator.

The m constraints specified by (2) define the feasible region in

the n dimensional Euclidean space in which the cost function

is required to be minimized [2].

Traditional methods for solving linear programming

problems typically involve an iterative process, but long

computational time limits their usage. There is an alternative

approach to solution of this problem.

Authors are with the Department of Electronics Engineering, Aligarh Muslim,

Aligarh, India (e-mail: [email protected]).

It is to exploit the artificial neural networks (ANN's) which

can be considered as an analog computer relying on a highly

simplified model of neurons [1]. ANN's have been applied to

several classes of constrained optimization problems and have

shown promise for solving such problems more effectively.

For example, the Hopfield neural network has proven to be a

powerful tool for solving some of the optimization problems.

Tank and Hopfield first proposed a neural network for

solving mathematical programming problems, where a

linear programming problem was mapped into a closed-loop

network [2]. Over the past two decades several neural-network

architectures for solving linear programming problems have

been proposed by Wang [3], Xia [4], Xia, Wang & Hung [5]

and Malek and Yari [6]. Wang‟s realization of a neural

network for solving linear programming problems utilizes 3n

amplifiers, n2+3n resistances and n capacitors. The circuit of

[4] requires 2m2+4mn amplifiers, 2m

2+4mn+3m+3 summers,

n+m integrators, and n simple limiters. The circuit of [5]

consists of m2+2mn amplifiers, 3n summers, n integrators, and

n simple limiters. And although no circuit implementation of

the proposed scheme is given in [6], it is evident that it would

not be simple to actually realize in hardware. Therefore, the

search still continues for a working circuit, to solve LPP,

which is feasible to implement in integrated forms and is fast

enough for utilization in real-time applications where the time-

to-solve is needed to be of the order of tenths of milliseconds

[5].

In this paper, a hardware solution to the problem of solving

a linear programming problem subject to linear inequality

constraints is presented. The proposed circuit employs only n

amplifiers, m DVCC-based unipolar comparators and

mn+m+n resistances. It is evident that the circuit complexity

of the proposed scheme is much reduced as compared to

existing ones [3-6].


proposed neural network to minimize a first order polynomial

in 2 variables subject to 2 linear constraints. Section III

contains results of PSPICE simulation. Some concluding

remarks appear in section IV.



(3)

Subject to

(4)

(5)

L

978-1-4244-8542-0/10/$26.00 ©2010 IEEE



minimize (3) subject to (4, 5) is presented in Fig. 1.

b1

s11Ix

a11V1+ a12V2

s12Ix

s21Ix

a21V1+ a22V2

s22Ix

+

_

V1

+_

Rp2C2V2

Rp1 C1

R1

R2

A

B

P1

P2

x1 Z+

1

X

Y2

Y1

Z+

2

x1 Z+

1

X

Y2

Y1

Z+

2

b2

DVCC-based

unipolar comparator

DVCC-based

unipolar comparator

Fig. 1 The proposed neural network for solving a linear programming problem in 2 variables subject to 2 linear constraints

As can be seen from Fig. 1, individual equations from the set


are realized using DVCC-based unipolar comparators.

x1

Z+

X

Y2

Y1

DVCC

Z-

VY1

VY2

VX

IX

IZ+

IZ-

Fig. 2 Symbolic Representation of DVCC

The symbolic diagram of the DVCC is shown in Fig. 2. The

device can be characterized by the following port relations:

[

]

[

]

[

]

(6)

Although both Z+ and Z- types of current outputs are

mentioned in (4), the DVCCs used in the proposed network

use only Z+ type of outputs. For the DVCC, unipolar

comparator action can be achieved by putting i.e. by

directly grounding the X-terminal [7] and operating the

DVCC from a unipolar biasing supply. For such a case, the

current in the X-port will saturate and can be written as

[ ( – ) ] (7)


very high) and are the saturated output current levels of

the comparator. Equation (7) can also be written in an

equivalent notation as

[ ( – ) ] (8)

where is the transconductance from the input voltage ports

(Y1 and Y2) to the output current port (X) and are the

biasing voltages of the DVCC-based unipolar comparator. By

virtue of DVCC action, this current will be transferred to the

Z+ ports as

[ ( – ) ] (9)

Where is the current scaling factor introduced during the

current conveying process from the X-port to the Z+ port.

Ideally, . However, by altering the aspect ratios of the

transistors used in the output stages of the DVCC, can be

made a scaled replica of . In that case, the value of will be

different from unity. This in property has been employed in

the design of the multi-output DVCCs needed for the

proposed circuit.

The outputs of the unipolar comparators are fed to neurons

having weighted inputs. These weighted neurons are realized

by using opamps where the scaled currents coming from

various unipolar comparators act as weights. Weighing is also

provided by the self-feedback resistances (R11 & R22 for

neurons 1 & 2 respectively) and the cross-feedback resistances

(R21 and R12 for neurons 1 & 2 respectively). Rpi and Cpi are

the input resistance and capacitance of the opamp

corresponding to the i-th neuron. These parasitic components

are included to model the dynamic nature of the opamp.

Node equation for nodes „A‟ and „B‟ give the equations of

motions of the first and second neurons as

[

] [

] (10)

where,

(11)

And,

[

] [

] (12)

where,

(13)

In equations (10) and (12) ui is the internal state of the i-th

neuron and is the current scaling factor at the j-th output of

i-th DVCC-based unipolar comparator. As is shown later in

this section, these weights are governed by the constraint

inequalities (4, 5). Using (8) in (10) and (12) results in (14)

and (15) given below.

[ ( – ) ]

[ ( – ) ] [

]

[

] (14)

[ ( – ) ]

[ ( – ) ] [

]

[

] (15)

978-1-4244-8542-0/10/$26.00 ©2010 IEEE

Moreover, it can be shown that the network of Fig. 1 may be

associated with an „energy‟ function of the form

( )

( – )

( – ) (16)


( )

( ) (17)

Also, if „E‟ is the Energy Function, it must satisfy the


(18)

Where „K‟ is a constant of proportionality and has the

dimensions of resistance. Substituting for

in (18) from

(14) and comparing with (17) yields

⁄

⁄ (19)

(20)

The values of the current scaling factors (sji) of Fig. 1 can

be easily calculated by choosing a suitable value of „K‟ and

then using (19). Analysis on similar lines can be performed to

obtain the values of the synaptic weights for the second

neuron and are presented in (21) and (22) below.

⁄

⁄ (21)

(22)




(23) subject to


are obtained from (20, 22). Towards that end, first the value of

was computed for the DVCC which was found to be 1.091

milli-mhos. Choosing ⁄ allows us to fix the current

scaling factors as given below:

The circuit for DVCC was taken from [7] and is reproduced in

Fig. 3 for reference. Standard 0.5 micron CMOS parameters

were used for simulation purposes. DVCC-based unipolar

comparators were obtained by operating the DVCCs using

+5V and 0.2V as VDD and VSS supply voltages respectively

instead of the ±2.5V supplies used in [7]. The bias voltage

VBB was kept at 0.05V.

Y1Y2

VBB

M1 M4M2 M3

M5 M6

M9 M10

M7

M11

M8

M12

XZ+

VSS

VDD

Fig. 3 CMOS implementation of the DVCC [7]

The DC transfer characteristics obtained are presented in Fig.

4 from which it may be observed that unipolar comparator

action is indeed realized. For the opamp, use was made of the

LMC7101A CMOS opamp from National Semiconductor.

The sub-circuit file for this opamp is available in Orcad Model

Library.

Fig. 4 DC transfer characteristics of the DVCC-based

unipolar comparator

Routine mathematical analysis of (23, 24) yields: = 0, =

1. The results of PSPICE simulation are presented in Fig. 5.

From the plots of the neuron output voltages, it can be seen

that V1 = - 9.97mV and V2 = 1.01 V which correspond closely

to the algebraic solution thereby confirming the validity of the

approach. The slight error in the obtained solutions may be

attributed to the non-ideal characteristics of the unipolar

comparators implemented using DVCCs, as depicted in Fig. 4.

(24)

978-1-4244-8542-0/10/$26.00 ©2010 IEEE


IV. CONCLUSION

In this paper, a CMOS compatible approach to solve linear

programming problem in 2 variables subject to 2 linear

constraints employing 2-neurons and 2-synapses, is described.

Each neuron requires one opamp and each synapse is

implemented using a differential transconductance element

realized using DVCCs. This results in significant reduction in

hardware over the existing schemes [3-6]. The proposed

network was tested on a sample problem of minimizing a

linear function in 2 variables and the simulation results

confirm the validity of the approach.

V. REFERENCES

[1] J.J. Hopfield, D.W. Tank, “„Neural‟ computation of decisions

optimization problems,” Biological Cybern., 52, 141-152, 1985

[2] D. W. Tank and J. J. Hopfield, “Simple Neural Optimization Networks: An A/D Converter, Signal Decision Circuit, and A Linear Programming

Circuit,” IEEE Trans. Circuits and Systems, Vol. CAS-33, No.5, pp. 533-541, May 1986.

[3] J. Wang, “Analysis and Design of a Recurrent Neural Network for

Linear Programming”, IEEE Trans. Cir and Sys–1: Fundamental Theory and Applications, Vol.40, No.9, pp. 613-618, Sept.1993.

[4] Y. Xia, “A New Neural Network for Solving Linear and Quadratic

Programming Problems,” IEEE Trans. Neural Net., Vol.7, No.6, pp. 1544-1547, Nov 1996.

[5] Y. Xia, J. Wang and D. L. Hung, “Recurrent Neural Networks for

Solving Linear Inequalities and Equations,” IEEE Trans. Circuits and Systems–1: Fundamental Theory and Applications, Vol.46, No.4, pp.

452-462, April 1999.

[6] A. Malek and A. Yari., “Primal–dual solution for the linear programming problems using neural networks,” Applied Mathematics

and Computation, 167, 198–211, 2005.

[7] S. Maheshwari, (2008) A canonical voltage-controlled VM-APS with a grounded capacitor,” Cir. Syst. Sig. Process, 27, 123-132.

[8] S. A. Rahman, Jayadeva, S. C. Dutta Roy, “Neural network approach to

graph colouring,” Electronics Letters, 35, 1173 – 1175, 1999.

publications from the thesis - shodhgangashodhganga.inflibnet.ac.in/bitstream/10603/13127/17/17...2....

Documents