artificial neural network (back-propagation neural network)

16
Artificial Neural Network (Back-Propagation Neural Network) Yusuf Hendrawan, STP., M.App.Life Sc., Ph.D

Upload: uta

Post on 23-Feb-2016

302 views

Category:

Documents


12 download

DESCRIPTION

Artificial Neural Network (Back-Propagation Neural Network). Yusuf Hendrawan , STP., M.App.Life Sc., Ph.D. http://research.yale.edu/ysm/images/78.2/articles-neural-neuron.jpg. http://faculty.washington.edu/chudler/color/pic1an.gif. Neurons. Biologica l. Artificial. A typical AI agent. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Artificial Neural Network (Back-Propagation Neural Network)

Artificial Neural Network(Back-Propagation Neural Network)

Yusuf Hendrawan, STP., M.App.Life Sc., Ph.D

Page 2: Artificial Neural Network (Back-Propagation Neural Network)

Neurons

http://faculty.washington.edu/chudler/color/pic1an.gif

http://research.yale.edu/ysm/images/78.2/articles-neural-neuron.jpg

Biological

Artificial

Page 3: Artificial Neural Network (Back-Propagation Neural Network)

A typical AI agent

Page 4: Artificial Neural Network (Back-Propagation Neural Network)

Neural Network Layers•Each layer receives its inputs from the previous layer and forwards its outputs to the next layer

http://smig.usgs.gov/SMIG/features_0902/tualatin_ann.fig3.gif

Page 5: Artificial Neural Network (Back-Propagation Neural Network)

Multilayer feed forward networkIt contains one or more hidden layers (hidden neurons).“Hidden” refers to the part of the neural network is not seen directly from either input or output of the network . The function of hidden neuron is to intervene between input and output.By adding one or more hidden layers, the network is able to extract higher-order statistics from input

Page 6: Artificial Neural Network (Back-Propagation Neural Network)

Neural Network LearningBack-Propagation Algorithm:

function BACK-PROP-LEARNING(examples, network) returns a neural network inputs: examples, a set of examples, each with input vector x and output vector y network, a multilayer network with L layers, weights Wj,i , activation function g

repeat for each e in examples do for each node j in the input layer do aj ‰ xj[e] for l = 2 to M do ini ‰ åj Wj,i aj

ai ‰ g(ini) for each node i in the output layer do Dj ‰ g’(inj) åi Wji Di

for l = M – 1 to 1 do for each node j in layer l do Dj ‰ g’(inj) åi Wj,i Di

for each node i in layer l + 1 do Wj,i ‰ Wj,i + a x aj x Di

until some stopping criterion is satisfied return NEURAL-NET-HYPOTHESIS(network)

[Russell, Norvig] Fig. 20.25 Pg. 746

Page 7: Artificial Neural Network (Back-Propagation Neural Network)

Back-Propagation Illustration

ARTIFICIAL NEURAL NETWORKS Colin Fahey's Guide (Book CD)

Page 8: Artificial Neural Network (Back-Propagation Neural Network)

X1

X2

Y

Vo Wo

Z1

Z2

Z3

Z4

Input (X) Hidden Output (Y)

Page 9: Artificial Neural Network (Back-Propagation Neural Network)

X1 X20.3 0.40.5 0.60.2 0.30.4 0.7

Input (X) Output / Target (T)

T0.10.80.40.5

Jumlah Neuron pada Input Layer 2Jumlah Neuron pada Hidden Layer 4Jumlah Neuron pada Output Layer 1Learning rate (α) 0.1Momentum (m) 0.9Target Error 0.01Maximum Iteration 1000

Page 10: Artificial Neural Network (Back-Propagation Neural Network)

Bobot Awal Input ke Hidden

V11 = 0.75

V21 = 0.35

V12 = 0.54

V22 = 0.64

V13 = 0.44

V23 = 0.05

V14 = 0.32

V24 = 0.81

Bias ke Hidden

Vo11 = 0.07 Vo21 = 0.12Vo12 = 0.91 Vo22 = 0.23Vo13 = 0.45 Vo23 = 0.85Vo14 = 0.25 Vo24 = 0.09

Bobot Awal Hidden ke Output

W1 = 0.04W2 = 0.95W3 = 0.33W4 = 0.17

Bias ke Output

Wo1 = 0.66Wo2 = 0.56Wo3 = 0.73Wo4 = 0.01

Page 11: Artificial Neural Network (Back-Propagation Neural Network)

Menghitung Zin & Z dari input ke hiddenZin(1) = (X1 * V11) + (X2 * V21) = (0.3 * 0.75) + (0.4 * 0.35) = 0.302Zin(2) = (X1 * V12) + (X2 * V22) = (0.3 * 0.54) + (0.4 * 0.64) = 0.418Zin(3) = (X1 * V13) + (X2 * V23) = (0.3 * 0.44) + (0.4 * 0.05) = 0.152Zin(4) = (X1 * V14) + (X2 * V24) = (0.3 * 0.32) + (0.4 * 0.81) = 0.42

575.01

11

1302.0)1( )1(

eeZ

inZ

603.01

11

1418.0)2( )2(

eeZ

inZ

538.01

11

1152.0)3( )3(

eeZ

inZ

603.011

11

42.0)4( )4(

ee

ZinZ

Page 12: Artificial Neural Network (Back-Propagation Neural Network)

Menghitung Yin & Y dari hidden ke outputYin = (Z(1) * W1) + (Z(2) * W2) + (Z(3) * W3) + (Z(4) * W4)

= (0.57 * 0.04) + (0.603 * 0.95) + (0.538 * 0.33) + (0.603 * 0.17)

= 0.876706.0

11

11

876.0

eeY

inY

Menghitung dev antara Y dengan output nyata

dev = (T - Y) * Y * (1 - Y) = (0.1 – 0.706) * 0.706 * (1 – 0.706) = -0.126Menghitung selisihselisih = T - Y= -0.606

Page 13: Artificial Neural Network (Back-Propagation Neural Network)

Back-Propagation

Menghitung din dari output ke hiddendin(1) = (dev * W1) = (-0.126 * 0.04) = -0.00504din(2) = (dev * W2) = (-0.126 * 0.95) = -0.1197din(3) = (dev * W3) = (-0.126 * 0.33) = -0.04158din(4) = (dev * W4) = (-0.126 * 0.17) = -0.02142

Menghitung dd (1) = (din(1) * Z(1) * (1 - Z(1) ) = (-0.00504 * 0.575 * (1 – 0.575) = -0.00123d (2) = (din(2) * Z(2) * (1 - Z(2) ) = (-0.1197 * 0.603 * (1 – 0.603) = -0.02865d (3) = (din(3) * Z(3) * (1 - Z(3) ) = (-0.04158 * 0.538 * (1 – 0.538) = -0.01033d (4) = (din(4) * Z(4) * (1 - Z(4) ) = (-0.02142 * 0.603 * (1 – 0.603) = -0.00512

Page 14: Artificial Neural Network (Back-Propagation Neural Network)

Mengkoreksi bobot (W) dan bias (Wo)

W1 = W1 + (α * dev * Z(1) ) + (m * Wo(1)) = 0.04 + (0.1 * -0.126 * 0.575) + (0.9 * 0.66) = 0.627 W2 = W2 + (α * dev * Z(2) ) + (m * Wo(2)) = 0.95 + (0.1 * -0.126 * 0.603) + (0.9 * 0.56) = 1.45 W3 = W3 + (α * dev * Z(3) ) + (m * Wo(3)) = 0.33 + (0.1 * -0.126 * 0.538) + (0.9 * 0.73) = 0.98 W4 = W4 + (α * dev * Z(4) ) + (m * Wo(4)) = 0.17 + (0.1 * -0.126 * 0.603) + (0.9 * 0.01) = 0.171 Wo1 = (α * Z(1) ) + (m * Wo(1)) = (0.1 * 0.575) + (0.9 * 0.66) = 0.65 Wo2 = (α * Z(2) ) + (m * Wo(2)) = (0.1 * 0.603) + (0.9 * 0.56) = 0.564 Wo3 = (α * Z(3) ) + (m * Wo(3)) = (0.1 * 0.538) + (0.9 * 0.73) = 0.71 Wo4 = (α * Z(4) ) + (m * Wo(4)) = (0.1 * 0.603) + (0.9 * 0.01) = 0.0693

Page 15: Artificial Neural Network (Back-Propagation Neural Network)

Mengkoreksi bobot (V) dan bias (Vo)

V11 = V11 + (α * d (1) * X1 ) + (m * Vo(11)) = 0.75 + (0.1 * -0.00123 * 0.3) + (0.9 * 0.07) = 0.8129 V12 = V12 + (α * d (2) * X1 ) + (m * Vo(12)) = 0.54 + (0.1 * -0.02865 * 0.3) + (0.9 * 0.91) = 1.3581V13 = V13 + (α * d (3) * X1 ) + (m * Vo(13)) = 0.44 + (0.1 * -0.01033 * 0.3) + (0.9 * 0.45) = 0.8446 V14 = V14 + (α * d (4) * X1 ) + (m * Vo(14)) = 0.32 + (0.1 * -0.00512 * 0.3) + (0.9 * 0.25) = 0.5448 V21 = V21 + (α * d (1) * X2 ) + (m * Vo(21)) = 0.35 + (0.1 * -0.00123 * 0.4) + (0.9 * 0.12) = 0.4579 V22 = V22 + (α * d (2) * X2 ) + (m * Vo(22)) = 0.64 + (0.1 * -0.02865 * 0.4) + (0.9 * 0.23) = 0.8458 V23 = V23 + (α * d (3) * X2 ) + (m * Vo(23)) = 0.05 + (0.1 * -0.01033 * 0.4) + (0.9 * 0.85) = 0.8145 V24 = V24 + (α * d (4) * X2 ) + (m * Vo(24)) = 0.81 + (0.1 * -0.00512 * 0.4) + (0.9 * 0.09) = 0.8907

Page 16: Artificial Neural Network (Back-Propagation Neural Network)

Mengkoreksi bobot (V) dan bias (Vo)

Vo11 = (α * d (1) * X1 ) + (m * Vo11) = (0.1 * -0.00123*0.3)+(0.9*0.07) = 0.0629Vo12 = (α * d (2) * X1 ) + (m * Vo12) = (0.1 * -0.02865*0.3)+(0.9*0.91) = 0.8181Vo13 = (α * d (3) * X1 ) + (m * Vo13) = (0.1 * -0.01033*0.3)+(0.9*0.45) = 0.4046Vo14 = (α * d (4) * X1 ) + (m * Vo14) = (0.1 * -0.00512*0.3)+(0.9*0.25) = 0.2248Vo21 = (α * d (1) * X2 ) + (m * Vo21) = (0.1 * -0.00123*0.4)+(0.9*0.12) = 0.1079Vo22 = (α * d (2) * X2 ) + (m * Vo22) = (0.1 * -0.02865*0.4)+(0.9*0.23) = 0.2058Vo23 = (α * d (3) * X2 ) + (m * Vo23) = (0.1 * -0.01033*0.4)+(0.9*0.85) = 0.7645Vo24 = (α * d (4) * X2 ) + (m * Vo24) = (0.1 * -0.00512*0.4)+(0.9*0.09) = 0.0807