neural network algorithm 2

8/6/2019 Neural Network Algorithm 2

http://slidepdf.com/reader/full/neural-network-algorithm-2 1/31



` Definition

` Neural Model

` Activation Function

`

Algorithm` Stage one conversion

` Stage two conversion

` Binary Neural implementation

`

Hardware Optimization` References



` Neural network is a sorted triple (N, V, w) with two sets N, V

and a function w

N is the set of neurons

V a sorted set

` {(i, j)|i, j E N} whose elements are called connections between

neuron i and neuron j.

` w : V R defines the weights

w((i, j)), the weight of the connection between neuron i and neuron j.



` Learning capa bility

` Predict the outcome of past trends.

`

Ro bust and Fault tolerant` Process information in parallel at high speed and in a

distri buted manner.



` Identity functionx F(x)=x for all x

` Binary Step function

x F(x)= 1 if x>=

= 0 if x<

` Sigmoid function

x F(x)=

` Bipolar sigmoid function

F(x)=



` Steps

Digitization

Conversion of digitized model into logic gate structure

Hardware optimization by elimination of redundant logic

gates



` Digitization of one Neuron Mathematical Model

` Real values between -1 and +1 can be represented by



` Functionality of the neuron should not be affected

while transforming analogue neuron into an

appropriate digital model.



` Conversion is achieved by transforming the analog input into

digital inputs

` Each analogue neuron input is transformed to its equivalent

group of n b binary inputs.

` Each input defined by initial weight wij into n b su binputs,

whose weights wijp ( p=0,1,« n b-1) is calculated as



` Argument corresponding to the neuron after the first conversion

stage is calculated as

= constant



` After 1st conversion neurons can have both positive and

negative weights.

`

Stage two aims to replace these neurons with equivalent oneshaving only positive weights.

` ie



` R elati s i bet ee t e i t bits s lie t sta e 2 e r s

a sta e e r s is i e b

` T ese t alter ati es ca be c resse i t



` Ar e ts f t e acti ati f cti bef re a after sta e 2

c ersi s l be e al

` ie

` T eref re



` Su bstituting the value of wijp



` Therefore threshold level of the stage two neurons is

` The neuron parameter after stage two can be calculated as a

function of initial analog neuron parameter



` Iterative implementation procedure uses 3 input parameters:

Index defining the current terminal group(F)

The current threshold level (T)

The logic gate type(LGT)

x ANY_GATE

x

AND_GATE



` At first step F=1,T=t(2) , LGT=ANY_GATE

1. If LGT=AND_GATE then go to 7., else 2.

2. Calculate the num ber of X input weights and determine num ber Y

of the cumulative weights which are >T.

If X>1 and Y=0 then go to3.

If X>1 and Y>0 then go to 4

If X=1 and Y=0 then go to 5

If X=0 and Y=1 then go to 7.

If X=0 and Y=0 then go to 6.



` The hardware implementation netlist o btained is repeatedly

analyzed and redundant logic gates with the same input

signals and are of the same type are removed.

` This optimization ends when no more gates can be removed.



[1] A. Dinu and M. N. Cirstea, ³A digital neural network FPGA direct

hardware implementation algorithm,´ in Proc. ISIE, Vigo, Spain, pp. 2307±

2312.

[2]Andrei Dinu, Marcian N. Cirstea ³Direct Nueral Network Hardware

Implementation Algorithm´ IEEE Transactions on Industrial Electronics

Vol 57,No 5, May 2010

[3] Martin T. Howard B Demuth and Mark Beale: Neural Network Design,

Vikas Thomas Learning

[4] Simon Haykin: Neural Networks- A Comprehensive Foundation, Pearson

Education

[5] Maurico A.Leon, James Keller ³Toward Implementation of Artificial

Neural Networks That "Really Work³´, Department of Computer

engineering and Computer Science,University of Missouri Colum bia.



` The novel algorithm treats each neuron is treated as a Boolean

functions with properties that can be ex ploited to achieve

compact implementation.

` Most efficient for low num ber of inputs on each input.



[6] Y. Singh, A. S. Chauhan, "Neural Networks in Data Mining," J ournal of

Theoretical and Applied Information Technology, vol 5. No 6, pp.37²42,

J une 2009

[7] C. M. Bishop, "Pattern Recognition and Machine Learning," New York:

Springer, 2006. ² 703 p.

[8] V. Ganapathy, K. L. Liew, "Handwritten Character Recognition Using

Multiscale Neural Network Training Technique," World Academy of Science,

Engineering and Technology, No 39, pp. 32²37, 2008.

[9] G. P. Zhang, "Neural Networks for Classification: A Survey," IEEE Trans.

on Syst., Man and Cybern, vol. 30. No 4, pp. 451²462, Nov. 2000.

[10] K. R. Farell, R. J. Mommone, K. T. Assaleh, "Speaker Recognition Using

Neural Networks and Conventional Classifiers," IEEE Transactions on

Speech and Audio Processing, vo2. 3. No 1, pp.194²205, 1994.

neural network algorithm 2

Documents