Artificial Neural Network

Learning Objective

The fundamentals of artificial neural network.The evolution of neural network.Comparison between biological neuron and

artificial neuron.

Artificial Neural Network

An efficient information processing system which resembles in characteristics with a biological NN.

Consists of processing elements known as neurons.

Information is contained in the form of weights associated with each connection link.

ANN is having the ability to learn, recall and generalize training patterns.

Each neuron has an internal state i.e. activation or activity level of neuron, which is the function of the inputs the neuron inputs.

Architecture of Simple ANN

x1

w1

y x2 w2

Yin=x1w1+x2w2

y=f(yin)

X1

Y

X2

Biological Neural Network

Human brain consists of a huge number of neurons approximately 1011.

Soma or Cell Body: where cell nucleus is located.Dendrites: where the nerve is connected to the cell body. Axon: which carries the impulses of the neuron

Terminology relationships b/w biological and artificial neurons

Biological Neuron

Artificial Neuron

Cell Neuron

Dendrites Weights or interconnections

Soma Net Input

Axon Output

Brain vs. Computer

SpeedProcessingSize and ComplexityStorage Capacity (memory)ToleranceControl Mechanism

Evolution of Neural Network

Basic Models of Neural Network

Three basic entities The model’s synaptic interconnection. The training or learning rules adopted for updating

and adjusting the connection weights. Their activation functions

Connections

There exist five basic types of neuron connection architectures: Single Layer feed forward network Multilayer feed forward network Single node with its own feedback Single layer recurrent network Multilayer recurrent network.

Learning

Parameter LearningStructure Learning

Supervised Learning Unsupervised Learning Reinforcement Learning

Activation Functions

Identity Function: f(x)=x for all xBinary Step Function:

f(x)=1 if x>=t =0 if x<t

Bipolar Step Function: f(x)=1 if x>=t

=-1 if x<tSigmoidal FunctionRamp Functions

f(x)= 1 if x>1x if o<=x<=10 if x<0

Important Terminologies of ANN

WeightsBiasThresholdLearning RateMomentum Factor: Vigilance Parameter: control the degree of

similarity required for patterns to be assigned to the same cluster unit.

Ex: For the network shown in figure, calculate the net input to the output neuron.

0.3 0.2

0.5 0.1 y

-0.3 0.6

x1

y

x3

x2

Ex. Calculate the net input for the network shown in figure with bias included in the network

0.2 0.45 0.3 y 0.7 0.6

x1

y

x2

Ex.: Obtain the output of the neuron y for the network shown in figure using activation functions as:1. binary sigmoidal2. bipolar sigmodial

0.8 0.35 0.1

0.6 0.3 y

-0.2 0.4