sổ tay cdt chuong 32-noron va hthg mo

8/6/2019 s tay cdt Chuong 32-Noron Va Hthg Mo

1/20

32Mng n ron v h thng m

Bogdan M. WilamowskiUniversity of Wyoming

32.1Mng n non v h thng m ..................................... ........1

32.2T bo n ron ................................................................. .....1

32.3Mng n ron truyn thng ...................................................3

32.4Cc thut ton hc cho mng n ron ...................................4

32.5Cc mng n ron truyn thng c bit .............................8

32.6Mng n ron hi qui ..........................................................1332.7 H thng m ....................................................................14

32.8 Thut ton di truyn ...................................................... ...17

32.1 Mng n non v h thng m

Nhng thit b in t hin i v ang thi thc cc nh nghin cu xy dng nhng c my thng minh hot ngtng t nh h thn kinh ca con ngi. S hp dn ca mc tiu ny c bt u khi McCulloch v Pitts pht trin mhnh n ron tnh ton c bn vo nm (1943) v khi Hebb gii thiu quy tc t hc vo nm (1949). Mt thp k sau (1958)Rosenblatt gii thiu khi nim perceptron. Vo u nhng nm 60 Widrow v Holf (1960, 1962) pht trin h thng thng

minh nh ADALINE v MADALINE. Vo nm (1965) trong cun sch Learning machines ng tng kt s pht trinca thi gian ny. Cc n bn nm (1969) ca Mynsky vi mt vi kt qu lm nn lng, lm gim mnh s hp dn camng n ron nhn to, v cc thnh tu trong c s ton hc ca thut ton lan truyn ngc ca Webos (1974) b b qua.S pht trin nhanh chng trong lnh vc mng n ron li bt u vi Hopfield (1982, 1984) vi mng hi quy, Kohonen(1982) vi cc thut ton hun luyn khng gim st, v mt s miu t ca thut ton lan truyn ngc ca Rumelhart et alvo nm 1986.

32.2 T bo n ron

N ron sinh hc l mt cu trc phc tp, nhn c s hun luyn t cc xung mch trn hng trm u vo kch thch(Excitatory) v kim ch (inhibitory). Cc xung ny c tnh tng vi cc trng s khc nhau (hoc tnh trung bnh) trongkhong thi gian ca tng kh nng (latent summation). Nu gi tr c tnh tng ln hn mt ngng, th n ron t pht ramt xung gi ti n ron ln cn. V cc xung n c tnh tng theo thi gian, nn n ron pht xung hun luyn tn s

cao hn i vi kch thch cao hn. Ngc li, nu tng ca cc gi tr u vo c trng s ln hn, th n ron pht cc xungnhanh hn. Ti cng mt thi im, mi n ron c m t bi s khng kch thch trong thi gian xc nh sau xung kch.Khong thi gian ny cng c gi l giai on tr (refactory period) c th c miu t chnh xc hn nh l mt hintng m sau khi kch thch gi tr ngng tng ln mt gi tr rt cao v sau gim t t vi mt hng s thi gianxc nh. Giai on tr thit lp cc gii hn mm trn tn s ca chui xung u ra. Vi n ron sinh hc, thng tin cgi di dng chui xung iu chnh.

Vic miu t hot ng ca n ron ny dn n mt m hnh n ron rt phc tp, n khng phi l n ron thc t. Vonm 1943 McCulloch v Pitts cho thy rng thm ch vi mt m hnh rt n gin, n c th xy dng cc mch nh vmch logic. Hn na, cc n ron n gin vi cc ngng ny thng mnh hn cc cng logic thng thng c s dngtrong my tnh. M hnh n ron McCulloch-Pitts gi thit rng cc tn hiu n v cc tn hiu i c th ch l gi tr nh phn0 v 1. Nu cc tn hiu n c gi thit c gi tr ln hn ngng th u ra n ron c t l 1. Ngc li, n c t l0.

1,0,

=

net T Tnet T

(32.1)

Trong T l gi tr ngng v gi tr netl tng trng s ca tt c cc tn hiu vo.

1


2/20

S tay C in t

1=

= n

i i

i

net w x (32.2)

Cc v d v n ron ca McCulloch-Pitts thc hin cc php OR, AND, NOT, v php NH (MEMORY) c minh hanh hnh 32.1. Ch rng cu trc ca cng OR v AND c th ging y ht nhau. Vi cc cu trc tng t, cc hm logickhc c th c thc hin, nh hnh 32.2. M hnh perceptron c cu trc tng t. Cc tn hiu vo, cc hm trng s vcc gi tr ngng ca n c th c cc gi tr m hoc dng. Thng thng, thay v s dng gi tr ngng bin i, mtu vo hng s vi trng s m hoc dng c thm vo mi n ron, nh hnh 32.2. Trong trng hp ny, gi tr ngnglun c t bng 0 v gi tr netc tnh nh sau:

1

1

+=

= +n

i i n

i

net w x w (32.3)

trong wn+1 c gi tr bng gi tr ngng yu cu v c du ngc li. Perceptron mt lp c s dng thnh cng gii quyt nhiu mu bi ton c in. Cc hm kch hot ngng cng c cho bi

1, 0sgn( ) 1( )

0, 02

+= = =


3/20

Mng n ron v h thng m

Cc hm kch hot lin tc cho php hun luyn da trn gradient ca mng nhiu lp. Cc hm kch hot in hnh cminh ha nh hnh 32.4. Trong trng hp ny khi n ron vi u vo ngng c s dng (Hnh 32.3(b)), tham s cth b kh t phng trnh (32.6) v (32.7) v dc ca p ng n ron c th c iu khin bng cch thay i trng s.V vy, khng thc s cn thit phi s dng n ron vi h s bin i.

Ch rng ngay c cc m hnh n ron vi cc hm kch hot lin tc th cng khc xa so vi n ron sinh hc, cc hotng vi cc chui xung iu chnh lin tc.

HNH 32.4 Cc hm kch hot in hnh: (a) hm n cc ngng cng, (b) hm lng cc ngng cng, (c) hm n cclin tc, (d) hm lng cc lin tc

HNH 32.5 Mt v d ca mng n ron truyn thng 3 lp, n cng i khi c hiu nh l mng lan truyn ngc

32.3 Mng n ron truyn thng

Mng n ron truyn thng ch cho tn hiu i theo mt hng. Hn na, hu ht cc mng n ron truyn thng c tchc theo cc lp. Mt v d ca mng n ron truyn thng 3 lp c minh ha nh hnh 32.5. Mng ny gm cc nt uvo, 2 lp n, v mt lp u ra.

Mt n ron n gin c th tch cc mu u vo thnh 2 nhm, v s tch ny l tuyn tnh. V d, vi cc mu cminh ha nh hnh 32.6, ng tch i qua trc x1 v x2 ti im 10x v 20x . ng tch ny c th c thc hin vi mt

n ron c trng s nh sau 110

1w = x , 2 201=w x . Trong trng hp tng qut vi n chiu, trng s l

1

0

11+= = i n

i

w wx

(32.8)

Mt n ron c th ch chia cc mu tch bit tuyn tnh. chn mt vng n chiu khng gian u vo, phi s dngnhiu hn n+1 n ron. Nu nhiu nhm u vo c chn, th s n ron trong lp n u vo phi theo quy tc nhn. Nu sn ron trong lp lp n khng b gii hn, th tt c cc bi ton phn loi c th c gii quyt s dng mng 3 lp. mt vd v mng n ron, phn loi 3 nhm trong khng gian 2 chiu, c minh ho nh hnh 32.7. Cc n ron trong lp n utin to ra cc ng ring bit gia cc nhm u vo.

3


4/20


5/20


Quy tc hc Instar

Nu cc vc t u vo v cc trng s c chun ha, hoc chng ch c 2 gi tr nh phn (1 hoc 0), th nets c gitr dng ln nht khi cc trng s v cc tn hiu vo ging nhau. Do , cc trng s ch phi thay i khi chng khc nhau.

( )i i iw c x wD = - (32.11)

Ch rng, thng tin c yu cu cho cc trng s ch c thc hin t cc tn hiu u vo. y l mt thut ton hcrt cc b v khng gim st.

Quy tc WTA (Winner Takes All)

WTA l mt s ci bin ca thut ton instar khi cc trng s ch c thay i vi cc n ron c gi tr netcao nht. Cctrng s ca n ron cn li khng thay i. i khi thut ton ny c thay i theo cch m ch mt s n ron c gi tr netcao nht c bin i ti mt thi im. Mc d y l mt thut ton khng gim st bi v chng khng bit u ra mongmun, nhng cng cn mt quan ta hoc ngi gim st tm mt n ron chin thng (winner) c gi tr net ln nht.Thut ton WTA c pht trin bi Kohonen (1982), thng c s dng cho cc nhm t ng v cho cc c tnh thngk sao chp ca d liu u vo.

Quy tc hc Outstar

Trong quy tc hc outstar, yu cu trng s c kt ni ti mt nt xc nh phi bng vi u ra mong mun cc n

ron lin kt ti cc trng s

( )ij i ijw c d wD = - (32.12)

trong jd l u ra n ron mong mun v c l hng s hc nh, n gim dn trong sut thc tc hc. y l quy tc hc cgim st bi v cc u ra mong mun phi c bit. C 2 quy tc hc instar v outstar u c pht trin bi Grossberg(1969).

Quy tc hc Widrow-Hoff

Vo nhng nm (1960, 1962) Widrow v Hoff pht trin mt thut ton hun luyn c gim st, n cho php hunluyn n ron vi p ng mong mun. Quy tc ny c bt ngun t bnh phng nh nht ca hiu gi tr netv gi tr ura.

2

1

( )p

j jp jp

p

Error net d =

= - (32.13)

trong

jError= l sai lch ca n ron th jP = s mu c cung cp

jpd = u ra mong mun ca n ron th j khi mu th p c cung cpnet= c cho bi phng trnh (32.2)

Quy tc ny cng c bit nh l quy tc bnh phng nh nht (LMS- Least mean square). Bng cch tnh o hm caphng trnh (32.13) i vi ijw , cng thc cho s thay i trng s c tm ra l:

1( )

p

ij i jp jp

pw cx d net

=D = - (32.14)

Ch rng s thay i trng s ijwD l tng ca cc s thay i t mi mu cung cp ring bit. V vy, c th sa litrng s sau khi mi mu ring bit c cung cp. Qu trnh ny c hiu nh l cp nht tng dn; cp nht tch ly l khicc trng s c thay i sau khi tt c cc mu c thc hin. Cp nht tng dn thng dn ti kt qu nhanh hn,nhng n nhy cm vi th t cc mu c thc hin. Nu hng s hc c c chn l nh, th tt c cc phng php chocng mt kt qu. Quy tc LMS lm vic tt vi tt c cc kiu hm kch hot. Quy tc ny c gng p buc gi tr netbngvi gi tr mong mun. i khi y khng phi l nhng g b quan st tm kim. Gi tr netbng bao nhiu khng phi liu quan trng nhng gi tr netm hay dng li l iu quan trng. V d, mt gi tr netln vi mt tn hiu ph hp sdn ti u ra ng v sai lch ln nh nh ngha bi phng trnh (32.13) v y c th l gii php hon ho.

Hi quy tuyn tnh

Quy tc hc LMS i hi yu cu hng trm hoc hng nghn php lp, s dng cng thc (32.14), trc khi n hi t tinghim ph hp. S dng quy tc hi quy tuyn tnh, kt qu ging nh th c th t c ch trong mt bc tnh.

5


6/20

S tay C in t

Xt mt n ron v s dng k hiu vc t cho mt b cc mu u vo X c cung cp thng qua vc t trng s w, vct gi tr netc tnh s dng:

Xw net = (32.15)

trong

X = l mng hnh vung ( 1)n xp+

n = l s u vop = l s mu

Ch rng kch thc ca cc mu u vo c tng thm mt, v trng s thm ny tng ng vi ngng (xem hnh332.3 (b)). Phng php ny, tng t nh quy tc LMS, gi thit mt hm kch hot tuyn tnh, v cc gi tr netphi bnggi tr u ra mong mun d.

Xw d = (32.16)

Thng thng 1p n> + , v phng trnh trc c th c gii ch trong trng hp sai lch bnh phng nh nht. Sdng i s vc t, nghim c cho bi:

1( )T Tw X X X d -= (32.17)

Khi phng php truyn thng c s dng, h p phng trnh vi n+1 cha bit, (32.16), phi c chuyn sang bn+1 phng trnh vi n+1 cha bit.

Y w z= (32.18)

khi cc thnh phn ca ma trn Y v vector z c cho bi

1 1

,p p

ij ip jp i ip p

p p

y x x z x d = =

= = (32.19)

Cc trng s c cho bi (32.17) hoc chng c th tm c bng nghim ca phng trnh (32.18).

Quy tc hc Delta

Phng php LMS gi s c hm kch hot tuyn tnh net=o, v gii php t c i khi l khng ti u, nh minh hatrn hnh 32.8 i vi trng hp hai chiu n, vi bn mu phn loi thnh hai nhm. Trong kt qu c c s dng thut

ton LMS, mt mu khng c phn loi.

HNH 32.8 V d vi s so snh kt qu t c s dng LMS v thut ton hun luyn delta. Ch rng LMS khng thtm c mt gii php thch hp

Nu sai lch c nh ngha l

2

1

( )p

j jp jp

p

Error o d =

= - (32.20)

Th o hm ca sai lch i vi trng s w ij l

1

( )2 ( )p jp jp jp ipij jp

dError df neto d xdw dnet =

= - (32.21)

6


7/20


t ta c o = f(net) v netc cho bi phng trnh 32.2. Ch rng o hm ny t l vi o hm ca hm kch hotf(net). V vy phng php ny ch c th p dng i vi hm kch hot lin tc v khng th s dng vi hm kch hotcng trong phng trnh (32.4) v (32.5). kha cnh ny, phng php LMS tng qut hn. Hm kch hot lin tc phbit nht c o hm l

(1 )f o o= - (32.22)

i vi trng hp n cc, phng trnh (32.6) v

20.5(1 )f o= - (32.23)

i vi trng hp lng cc, phng trnh (32.7)

S dng phng php tch ly, trng s n ron w ij s c thay i vi hng ca gradient

1

( )p

ij jp jp jp

p

w cx d o f =

D = - (32.24)

Trong trng hp hun luyn tng dn i vi mi mu c p dng

( )ij i j j jw cx f d oD = - (32.25)

s thay i trng s t l vi tn hiu u vo x i, s khc nhau gia u ra mong mun v thc t l djp - ojp v o hm cahm kch hot l fjp. Tng t nh lut LMS, trng s c th c cp nht trong c phng php tch ly v tng dn. Sosnh vi lut LMS, lut delta lun dn ti mt gii php ti u. Nh minh ha trn hnh 32.8, khi lut delta c s dng, ttc bn mu u c phn loi chnh xc.

Hc lan truyn ngc sai lch

Quy tc hc delta c th c tng qut ha cho cc mng nhiu lp. S dng phng php tng t vi quy tc hcdelta, gradient ca sai lch tng th c th c tnh ton i vi mi trng s trong mng.

ij i j jw cx f ED = (32.26)

Trong

c = hng s hc

xi = tn hiu trn u vo n ron th ifj = o hm ca hm kch hot

HNH 32.9 Minh ha khi nim tnh ton h s trong mng n ron

Sai lch tch ly Ej trn u ra n ron c cho bi

1

1( )

K

j k k jk

kj

E o d Af =

=

(32.27)

Trong K l s u ra mng v Ajk l mt khuch i tn hiu nh t u vo ca n ron th j ti u ra th k ca mng,nh minh ha trn hnh 32.9. S tnh ton sai lch lan truyn ngc bt u ti lp u ra v sai lch tch ly c tnh toncho tng lp n lp u vo. Phng php ny kh thc hin c trn phn cng. Do vy, mt phng php n gin hn

7


8/20

S tay C in t

l tm h s khuch i tn hiu t u vo ca n ron th j ti mi u ra mng nh minh ha trn hnh 32.9. Trong trnghp ny, trng s c hiu chnh s dng

1

( )K

ij i k k jk

k

w cx o d A=

= (32.28)

Ch rng cng thc ny l tng qut, vi mt s quan tm l n ron ny c sp xp trong cc lp hay khng. Mtcch tm h s khuch i Ajk l a ra mt s thay i tng dn trn u vo ca n ron th j v quan st s thay i

u ra th k ca mng. Th tc ny yu cu ch vi lan truyn tn hiu thun, v n d dng thc hin trn phn cng. Mtcch khc l tnh ton h s khuch i trn mi lp v tm tng ca chng. Cch ny c cng tnh ton t hn so vi tnhton sai lch tch ly trong thut ton lan truyn ngc sai lch.

Thut ton lan truyn ngc c xu hng dao ng. lm nhn qu trnh, trng s tng thm w ij c th c hiuchnh theo Rumelhart, Hinton, v Wiliams (1986):

( 1) ( ) ( ) ( 1)ij ij ij ijw n w n w n w n+ = + + (32.29)

Hoc theo Sejnowski v Rosenberg (1987)

( 1) ( ) (1 ) ( ) ( 1)ij ij ij ijw n w n w n w n + = + + (32.30)

Trong l s hng xung lng (momentum).

HNH 32.10 Minh ha tnh ton o hm hiu chnh cho s hi t nhanh ca thut ton lan truyn ngc sai lch

Thut ton lan truyn ngc c th c tng tc rt nhanh khi tm c thnh phn ca gradient, trng s c hiuchnh dc theo hng gradient n tn khi gi tr nh nht t c. Qu trnh ny c th c tip tc m khng cn tnhton hng gradient ti mi bc. Khi nhng thnh phn gradient mi c tnh ton ch mt ln, mt gi tr nh nht cc theo hng ca gradient lin trc. Qu trnh ny ch c th c p dng i vi s hiu chnh trng s tch ly. Mtphng php tm gi tr nh nht dc theo hng gradient l qu trnh leo cy tm li (tree step process of finding error)i vi ba im dc theo hng gradient v sau s dng xp x parabol, nhy trc tip ti gi tr nh nht. Thut ton hcnhanh s dng phng php miu ta c sut bi Fahlman (1988) v n c bit n nh l quickprop.

Thut ton lan truyn ngc c nhiu nhc im dn n vic hi t rt chm. Mt trong nhng nhc im ln nhtca thut ton lan truyn ngc l qu trnh hc hu nh b hng i vi cc n ron p ng li kt qu sai ton din. V d,nu gi tr u ra n ron gn vi +1 v u ra mong mun gn vi -1 th h s khuch i n ron f(net)= 0 v tn hiu sailch khng th truyn ngc v v vy th tc hc khng hiu qu. vt qua s kh khn ny, mt phng php sa i

cho vic tnh o hm c gii thiu bi Wilamowski v Torvik (1993). o hm c tnh ton nh dc ca mt ngthng kt ni im gi tr u ra vi im gi tr mong mun, minh ha trn hnh 32.10.

moddesired actual

if

desired actual

o of

net net

=

(32.31)

Ch rng i vi sai lch nh, phng trnh (32.31) hi t ti o hm ca hm kch hot ti gi tr u ra. Vi s tngdn chiu ca h thng s gim dn cc tiu cc b. Dn n hin tng miu t, nh mt ci by cc tiu cc b lnguyn nhn ca vn hi t trong thut ton lan truyn ngc sai lch.

32.5 Cc mng n ron truyn thng c bit

Mng lan truyn ngc nhiu lp, nh trn hnh 32.5 thng s dng mt mng truyn thng. Mng gm nhiu n ron

vi hm kch thch lin tc kiu sigmoid c th hin trn hnh 32.4(c) v 32.4(d). Trong hu ht cc trng hp, ch c cclp n c yu cu v s n ron trong mt lp n c la chn t l vi phc tp ca bi ton. S lng n ron trongmt lp n thng thng c tm bi qu trnh m (qu trnh th v sai). Qu trnh hun luyn c bt u vi tt c cc

8


9/20


trng s c la chn ngu nhin ti gi tr nh nht v s dng thut ton lan truyn ngc sai lch tm ra p n. Khiqu trnh hc khng hi t, s hun luyn c lp li vi mt b cc trng s c la chn ngu nhin mi.

Nguyen v Widrow (1990) xut mt phng php thc nghim cho gi tr ban u ca trng s mng hai lp. Tronglp th hai, trng s c la chn ngu nhin trong di t -0.5 ti +0.5. Trong lp th nht, trng s ban u c tnh t

( 1), ( , )ij

ij n j

j

zw w random

z

+= = + (32.32)

Trong zij l s ngu nhin t -0.5 ti +0.5 v h s t l c cho bi

1/0.7 Np = (32.33)

Trong n l s u vo v N l s n ron n trong lp th nht. Vi cc gi tr ban u ca trng s ny thng dn nkt qu nhanh hn.Trong nhng gii php y vi mng lan truyn ngc, nhiu c gng tiu biu c yu cu vi cutrc mng khc nhau v trng s ngu nhin ban u khc nhau. Mt iu quan trng l mng c hun luyn t ti mttnh cht tng quan. iu c ngha rng mng c hun luyn cng c th x l chnh xc cc mu m khng c sdng cho hun luyn. Do vy trong th tc hun luyn, thng mt s d liu c ly ra t mu hun luyn v cc mu nyc s dng cho vic kim tra. Nhng kt qu vi mng lan truyn ngc thng ph thuc vo may ri. iu ny khchl cc nh nghin cu pht trin mng truyn thng, n c tin cy cao hn. Mt s mng truyn thng c miu t trongcc phn sau.

Mng lin kt hm (Functional Link Network)Mng n ron mt lp n gin khi hun luyn, nhng nhng mng ny ch c th gii quyt cc bi ton tuyn tnh ring

l. Mt gii php c kh nng ng dng cho cc bi ton phi tuyn c trnh by bi Nilsson (1965) v c pht trinthnh cng bi Pao (1989) s dng mng lin kt hm nh trn hnh 33.11. S dng nhng s hng phi tuyn vi cc hmxc nh ban u, s lng u vo thc t c cung cp cho mng n ron mt lp c tng ln. Trong trng hp ngin nht, cc yu t phi tuyn l nhng s hng bc cao ca cc mu u vo. Ch rng mng lin kt hm c th c xl nh mng mt lp, trong d liu u vo tng thm c pht ra offline s dng cc bin i phi tuyn. Th tc hccho mng mt lp d dng v nhanh. Hnh 32.12 minh ha bi ton XOR c gii quyt s dng mng lin kt hm. Ch rng s dng phng php lin kt hm, bi ton kh ny tr nn n gin. Vn vi mng lin kt hm l vic la chnyu t phi tuyn thch hp khng phi l mt nhim v n gin. Trong nhiu trng hp thc t, tuy nhin n khng kh d on kiu bin i ca d liu u vo c th tuyn tnh ha vn v do vy phng php lin kt hm c s dng.

HNH 32.11 Mng lin kt hm

HNH 32.12 Mng lin kt hm cho gii php ca bi ton XOR: (a) s dng tn hiu n cc,(b) s dng tn hiu lng cc

9


10/20

S tay C in t

HNH 32.13 Mng lan truyn ngc

Kiu truyn thng ca mng lan truyn ngc (Feedforward version of the counterpropagation network)

Mng lan truyn ngc c xut u tin bi Hecht-Nilsen (1987). Trong phn ny, s tho lun mt kiu truynthng sa i nh c miu t bi Zurada (1992) . Mng ny c minh ha trn hnh 32.13, i hi s n ron n bngs mu u vo hoc chnh xc hn l s nhm u vo. Lp th nht c xem nh l lp Kohonen vi cc n ron n cc.Trong lp ny, ch c mt n ron, n ron chin thng, c tch cc. Th hai l lp outstart Grossberg. Lp Kohonen c thc hun luyn trong ch khng gim st nhng ci khng cn quan tm. Khi mu u vo nh phn c xem xt,trng s u vo phi chnh xc bng mu u vo. Trong trng hp ny

[ 2 ( , )]net x w n HD x w= = (32.34)

Trong n = s lng u vow = trng sx = vc t u vo

HD(w,x) = khong cch Hamming gia mu v trng s.

Cho mt n ron lp u vo ch phn ng vi mu c tch tr, gi tr ngng i vi n ron ny l( 1) ( 1)nw n+ = (32.35)

Nu yu cu cc n ron cng phi phn ng vi nhng mu tng t, th ngng nn l wn+1 = -[n-(1+HD)], trong HDl khong cch Hamming nh ngha mt di ging nhau. Do mu u vo c cho ch c mt n ron lp th nht c thc gi tr 1 v nhng n ron cn li c gi tr 0, nn trng s lp u ra bng mu u ra c yu cu.Mng vi hm kchhot n cc lp th nht lm vic nh mt bng tra cu. Khi hm kch hot tuyn tnh (hoc khng c hm kch hot no)c s dng trong lp th hai th mng cng c th c xem nh l mt b nh tng t. i vi mt a ch c pdng u vo nh mt vc t nh phn, mt tp hp cha ng nhng gi tr tng t nh cc trng s trong lp th hai cth c phc hi chnh xc. Mng lan truyn ngc truyn thng cng c th s dng cc u vo tng t nhng trongtrng hp ny tt c d liu u vo phi c chun ha

ii i

i

x

w x x= =

(32.36)

Mng lan truyn ngc rt d thit k. S n ron trong lp n bng s lng mu (nhm). Trng s lp u vo bngmu u vo v trng s lp u ra bng mu u ra. y l mng n gin c th c s dng cho vic to mu nhanh.Mng lan truyn ngc thng c nhiu n ron n hn yu cu. Tuy nhin, mt s n ron n d tha cng c s trongmng truyn thng phc tp hn nh mng n ron xc sut (PNN) Specht (1990) hoc mng n ron hi qui tng qut(GRNN) Specht (1992).

Kin trc WTA

Mng WTA (winner take all) c xut bi Kohonen (1988). y l mng mt lp c bn c s dng trong thutton hun luyn khng gim st rt ra tnh cht thng k ca d liu u vo, hnh 32.14 (a). bc u tin, tt c dliu u vo c chun ha do vy di ca mi vc t u vo l nh nhau v thng bng 1, phng trnh 32.36. Hm

tch cc ca n ron l lng cc v lin tc. Th tc hc bt u vi mt gi tr ban u ca trng s ti nhng gi tr ngunhin nh. Trong sut qu trnh hc, trng s c thay i ch vi cc n ron c gi tr ln nht u ra n ron chinthng.

10


11/20


( )w ww c x w = (32.37)

Trong

ww = trng s ca n ron chin thngx = vc t u voc = hng s hc

Thng thng, y l mng mt lp c t chc thnh dng lp hai chiu , nh trn hnh 32.14(b). Hnh su cnh

thng c chn t c s giao nhau gia cc n ron. Cng vy, thut ton c sa i khng ch cc n ronchin thng m nhng n ron ln cn cng c php thay i trng s. Trong mi trng hp, hng s thi gian c phngtrnh 32.37 gim i vi mt khong cch tnh t n ron chin thng. Sau th tc hun luyn khng gim st, lp Kohonen cth t chc d liu thnh tng nhm. u ra ca lp Kohonen c kt ni ti mng truyn thng mt lp hoc hai lp vithut ton lan truyn li. y l s t chc d liu ban u trong lp WTA, thng dn n hun luyn nhanh ca nhng lptheo sau.

HNH 32.14 Kin trc WTA i vi nhm rt ra trong ch hun luyn khng gim st:(a) kt ni mng, (b) mng mt lp c t chc thnh hnh su cnh

HNH 32.15 Kin trc tng quan tng

Kin trc tng quan tng (Cascade Correlation Architecture)

Kin trc tng quan tng c xut bi Fahlman v Lebiere (1990). Qu trnh xy dng mng bt u vi mng nron mt lp v n ron n c thm vo khi cn thit. Kin trc mng c minh ha trn hnh 32.15. mi bc hunluyn, mt n ron n c thm vo v trng s ca n c iu chnh ti gi tr ln nht ca bin tng quan gia ura n ron n mi v tn hiu sai lch d u ra mng c loi tr. Tham s tng quan S phi t c cc i:

1 1

( )( )po

p po o

o p

S V V E E = =

= (32.38)

Trong o = s lng u ra mngp = s lng mu hun luynvp = u ra trn n ron n mi

11


12/20

S tay C in t

epo = li trn u ra mng

V v E0 l gi tr trung bnh ca Vp v Epo. Bng cch tm ra gradient, i/ wS , trng s, s iu chnh cho cc n ronmi c th tm c nh sau

1 1

( )po

i o po o p ip

o p

w E E f x= =

= (32.39)

Trong

0 = tn hiu tng quan gia gi tr u ra n ron mi v u ra n ron.

'

pf = o hm ca hm tch cc cho mu p

ipx = tn hiu u vo

N ron u ra c hun luyn s dng thut ton Delta hoc Quickprop. Mi n ron n c hun luyn ch mt ln vsau trng s ca chng c c nh. Qu trnh xy dng v hc mng c hon thnh khi kt qu thng k t c.

Mng chc nng c s ta trn (Radial Basis Function Networks, RBFN)

Cu trc ca RBFN c th hin trn hnh 32.16. Kiu mng ny thng ch c mt lp n vi cc n ron c bit. Min ron ny p ng ch vi tn hiu u vo gn vi mu tch tr. Tn hiu u ra hi ca n ron n th i c tnh s dng

cng thc2

2exp

2

i

i

x sh

=

(32.40)

HNH 32.16 Mt cu trc tiu biu ca RBFN

Trong

x = vc t u vosi = mu tch tr th hin tm ca nhm ii = bn knh ca nhm

Ch rng cch hot ng ca n ron ny khc nhiu so vi n ron sinh hc. Trong n ron ny, s kch thch khng

phi l mt hm ca tng trng s ca tn hiu u vo. thay th, khong cch gia u vo v mu lu tr c tnh ton.Nu khong cch ny bng khng, n ron phn ng vi gi tr u ra ln nht bng mt. N ron ny c kh nng nhn dngmt mu no v pht ra cc tn hiu u ra vi chc nng tng t. c im ca n ron ny l c tc ng mnh hn nron c s dng trong mng lan truyn ngc. Nh mt h qu, mng to ra t nhng n ron ny cng c tc ng mnhhn. Nu cc tn hiu u vo ging nhau nh mt mu tch tr trong mt n ron, th n ron ny a ra kt qu l 1 v cc nron cn li c gi tr 0 u ra, nh minh ha trn hnh 32.16. V vy tn hiu u ra ng bng vi trng s ca n ron tchcc. Theo cch ny, nu s lng n ron trong lp n m ln th nh x u vo - u ra c th t c. Tht khng may,vn c th xy ra i vi mt s mu c nhiu n ron trong lp th nht s a ra mt tn hiu khc khng. i vi mt sxp x thch hp, tng ca tt c cc tn hiu t lp n s bng 1. t c yu cu ny, tn hiu u ra thng c chunha, nh minh ha trn hnh 32.16.

Mng c bn ta trn c th c thit k v hun luyn. Vic hun luyn thng c tin hnh theo hai bc. Trongbc th nht, lp n c hun luyn ch khng quan st bng cch la chn mu tt nht cho vic th hin nhm. Sdng mt phng php tng t nh kin trc WTA. Cng trong bc ny, bn knh i phi c tm sao cho trng khp vi

cc nhm tng ng.Bc hun luyn th hai l thut ton lan truyn ngc sai lch c tin hnh ch cho lp u ra. V th y l thut ton

quan st ch cho mt lp, vic hun luyn rt nhanh, nhanh gp 100-1000 ln so vi mng nhiu lp lan truyn ngc. iu

12


13/20


ny to cho mng chc nng c bn ta trn rt hp dn. Cng vy, mng ny c th thit lp d dng s dng my tnh; tuynhin thc hin trn phn cng l rt kh.

32.6 Mng n ron hi qui

Tri ngc vi mng n ron truyn thng, u ra ca mng n ron hi qui c th c kt ni vi u vo ca chng. Dovy, tn hiu trong mng c th c quay vng lin tc. Cho n by gi, ch c mt s hu hn cc mng n ron hi qui

c miu t.

Mng Hopfield

Mng hi qui mt lp c phn tch bi Hopfield (1982). Mng ny c th hin trn hnh 32.17, c nhng n ronlng cc ngng cng vi u ra bng 0 hoc 1. Trng s c cho bi mt ma trn vung i xng W vi cc yu t m(Wij = 0 vi i=j) trn ng cho chnh. S n nh ca h thng thng c phn tch bi iu kin ca hm nng lng.

1 1

1

2

N N

ij i j

i j

E W v v= =

= (32.41)

C th chng minh rng trong sut qu trnh lu thng nng lng E ca mng gim v h thng hi t ti im n nh.iu ny c bit ng khi gi tr u ra ca h thng c cp nht trong ch khng ng b. iu ny c ngha rng timt chu k cho, ch c mt u ra ngu nhin c th c thay i ti gi tr yu cu. Hopfield cng chng minh rng

nhng im n nh lm cho h thng hi t c th c lp trnh bng cch iu chnh trng s s dng lut Hebbian sai.

(2 1)(2 1)ij ji i jw w v v c = = (32.42)

Do b nh c kh nng cha hu hn. Da trn thc nghim, Hopfield c lng gi tr ln nht ca mu c tchtr l 0.15N, trong N l s lng n ron.

HNH 32.17 Mng Hopfield hay l b nh kt hp t ng

Sau ny khi nim hm nng lng c m rng bi Hopfield (1984) cho mng hi qui mt lp c n ron l hm tchcc lin tc. Nhng kiu mng ny c s dng gii quyt cc vn lp trnh tuyn tnh v ti u.

B nh kt hp t ng

Hopfield (1984) m rng khi nim mng Hopfield thnh b nh kt hp t ng. Trong cu trc mng ging nhaunh trn hnh 32.17, nhng n ron ngng cng lng cc c s dng vi u ra bng -1 hoc +1. Trong mng ny,ngng sm c tch tr thnh ma trn trng s W s dng thut ton kt hp t ng

1

MT

m m

m

W s s MI =

= (32.43)

Trong M l s lng mu tch tr v I l ma trn n v. Ch rng W l ma trn vung i xng vi cc thnh phntrn ng cho chnh l 0 (Wij =0 vi i=j). S dng cng thc sa i (32.42), nhng mu mi c th c thm vo hoctr i t b nh. Khi b nh bc l mt mu lng cc nh phn bng cch rng buc nhng trng thi mng ban u, sau khilu chuyn tn hiu mng s hi t ti mu c tch tr gn nht (ging nht) hoc phn b ca n. im n nh ny s

gi tr cc tiu gn nht ca nng lng1

( )2

TE v v Wv= (32.44)

13


14/20

S tay C in t

Nh mng Hopfield, b nh kt hp t ng gii hn kh nng cha, c nh gi bi Mmax=0.15N. Khi s lng ccmu tch tr ln v gn ti kh nng ca b nh, mng c xu hng hi t ti trng thi khng xc thc, khng c tch tr.Trng thi khng xc thc ny l nhng cc tiu thm vo ca hm trng lng.

HNH 32.18 Mt v d ca b nh kt hp t ng hai chiu: (a) c miu t nh mng hai lp vi tn hiu tun hon, (b)c miu t nh mng hai lp vi lu lng tn hiu hai chiu

B nh kt hp hai chiu (BAMs)

Khi nim b nh kt hp t ng c m rng thnh b nh kt hp hai chiu (BAM) bi Kosko (1987, 1988). B nhny nh trn hnh 32.18 c kh nng kt hp mt cp mu a v b. y l mng hai lp vi u ra ca lp th hai c kt nitrc tip ti u vo ca lp th nht. Ma trn trng s ca lp th hai l WT v W cho lp th nht. Ma trn trng s vungW t c nh l tng ca cc ma trn tng quan cho (cross-correlational matrices)

1

M

m m

m

W a b=

= (32.45)

Trong M l s lng cp c tch tr, am v bm l cc cp vc t tch tr. Nu cc nt a v b c bt u vi mt vct tng t nh vc t tch tr th sau s lu thng tn hiu, c nhng mu tch tr am v bm c th c phc hi. BAMgii hn kh nng ca b nh v cc vn h hng b nh tng t nh b nh kt hp t ng. Khi nim BAM c thc m rng cho s kt hp ba hoc nhiu hn s vc t.

32.7 H thng mNhng ng dng quan trng ca mng n ron lin quan ti s nh x phi tuyn ca bin u vo n chiu ti bin u ra m

chiu. Nh mt chc nng c yu cu trong h thng iu khin, i vi nhng bin c o l xc nh, bin iukhin phi c pht ra. Mt cch tip cn khc cho s nh x phi tuyn ca mt tp cc bin thnh mt tp cc bin khc lb iu khin m. Nguyn tc hot ng ca b iu khin m khc hn vi mng n ron. S khi ca b iu khin mc th hin trn hnh 32.19. bc th nht, u vo tng t c bin i thnh tp cc bin m. Trong bc ny, ivi mi u vo tng t, 3-9 bin m tiu biu c pht ra. Mi bin m c mt gi tr tng t gia 0 v 1. bc tiptheo, logic m c p dng cho cc bin m u vo v mt tp kt qu cc bin u ra c pht ra. bc cui cng, ls gii m, t mt tp cc bin m u ra, mt hoc nhiu cc bin tng t c sinh ra, c s dng nh l nhng biniu khin.

HNH 32.19 S khi ca b iu khin m

14


15/20


HNH 32.20 Qu trnh m ho: (a) hm lin thuc tiu biu cho qu trnh m ho v gii m,

(b) v d v s bin i nhit thnh cc bin m

M ho

Mc ch ca s m ho l bin i mt bin tng t u vo thnh mt tp cc bin m. i vi chnh xc caohn, nhiu bin m s c la chn. minh ho qu trnh m ho, xem xt mt bin m l nhit v m ho thnh 5bin m: cold (lnh), cool (mt), normal (bnh thng), warm (m), v hot (nng). Mi bin m c gi tr gia 0 v 1, miu tmc kt hp ca u vo tng t (nhit ) trong bin m cho. i khi, s thay th thut ng mc kt hp thnhmc lin thuc c s dng. Qu trnh m ho c minh ho trn hnh 32.20. S dng hnh 32.20 chng ta c th tmra mc kt hp ca mi bin m vi nhit cho. V d, i vi nhit l 57 0F, tp cc bin m sau y t c: [0,0.5, 0.2, 0, 0], v i vi T = 800F l [0, 0, 0.25, 0.7, 0]. Thng ch c mt hoc hai bin m c gi tr ln hn khng. Trongv d ny, s dng hm hnh thang tnh ton mc kt hp. Cc hm khc nh tam gic hoc Gaussian cng c th cs dng min l gi tr tnh ton c nm trong di t 0 n 1. Mi hm lin thuc c miu t bi ch ba hoc bn tham

s c th c lu trong b nh.i vi s thit k trng thi m ho ring bit, cc lut thc hnh sau nn c s dng:

- Mi im ca bin tng t u vo nn ph thuc t nht l mt v khng nhiu hn hai hm lin thuc.

- i vi nhng hm chng, tng ca hai hm lin thuc phi khng ln hn mt. iu ny cng c ngha rng schng phi khng giao vi nhng im c gi tr cc i.

- i vi chnh xc cao hn, nhiu hm lin thuc nn c s dng. Tuy nhiu, rt nhiu hm dy c dn nphn ng ca h thng khng n nh.

nh gi lut

Tri ngc vi logic nh phn, cc bin c th ch c trng thi nh phn, trong logic m tt c cc bin phi c gi trgia 0 v 1. Logic m c cc ton t c bn ging nhau: ^ - AND, OR, v NOT:

A^B^C => min{A, B, C} gi tr nh nht ca A hoc B hoc CA B C => m ax{A, B, C} gi tr ln nht ca A hoc B hoc CA => 1-A

HNH 32.21 Bng m: (a) bng vi lut m, (b) bng vi cc bin trung gian t ij

HNH 32.22 S minh ho cho qu trnh gii m

V d, 0.1 ^ 0.7 ^ 0.3 = 0.1, 0.1 0.7 0.3 = 0.7, v 0.3 = 0.7. Nhng lut ny cng c hiu nh lut AND, OR, NOT(Zadel, 1965). Ch rng nhng lut ny cng ng i vi logic nh phn c in. Lut suy din c ch ra trong bng mnh trn hnh i vi h thng cho. Xt mt h thng n gin vi hai bin u vo tng t x v y v mt bin u ra z.

15


16/20

S tay C in t

Mc ch l thit k h thng m sinh ra z nh f(x,y). Sau khi m ho, bin tng t x c miu t bi 5 bin m: x 1, x2, x3,x4, x5 v bin tng t y c miu t bi 3 bin m: y 1, y2, y3. Gi s rng bin u ra tng t c miu t bi 4 bin m:z1, z2, z3, z4. Vn trng tm ca qu trnh thit k l thit lp cc bin m u ra thch hp (z k) cho tt c s kt hp cc binm u vo nh trn bng trong hnh 32.21. Ngi thit k phi ch ra nhiu lut nh: nu u vo c miu t bi bin mxi v yj th u ra c miu t bi bin m zk. Trn bng m c ch ra, tnh ton logic m tin hnh trong hai bc.Th nht, mi mt ca bng m c in vi bin m trung gian tij thu c t hot ng AND tij = min(x i, yj), nh trnhnh 32.21 (b). Bc ny c lp vi cc lut c yu cu i vi h thng cho. bc th hai, hot ng OR (max)c s dng tnh ton cho mi bin m u ra z k. Trong v d cho trn hnh 32.12, z1 = max{t11, t12, t21, t41, t51}, z2 =max {t13, t31, t42, t52}, z3 = max {t22, t23, t43}, z4 = max{t32, t34, t53}. Ch rng cc cng thc ph thuc vo nhng thng sc cho trong bng m trn hnh 32.21.

Gii m

Nh mt kt qu ca nh gi lut m, mi bin u ra lin tc c th hin bi nhiu bin m. Mc ch ca gii m lc c u ra lin tc. iu ny c th c thc hin bng cch s dng hm lin thuc nh trn hnh 32.20. Trong bcth nht, cc bin m thu c t s nh gi lut c s dng sa i hm lin thuc theo cng thc

* ( ) min{ ( ), }k k kz z z = (32.46)

V d, nu cc bin m u ra l 0, 0.2, 0.7, 0.0, th hm lin thuc c sa i c hnh dng nh ng m nh trnhnh 32.22. Gi tr tng t ca bin z c xc nh nh l trng tm ca hm lin thuc c sa i * ( )k z ,

*

1

log*

1

( )

( )

n

kk

ana n

kk

z zdzz

z dz

+

= +

=

=

(32.47)

Trong trng hp hnh dng ca hm lin thuc u ra ( )k z ging nhau, phng trnh trn c th c n gin hathnh

1log

1

n

k kkana n

kk

z zcz

z

=

=

=

(32.48)

Trong

n = s hm lin thuc ca bin u ra analogz

zk = bin u ra m thu c t s nh gi theo lutzck = gi tr tng t tng ng vi tm ca hm lin thuc th k

Phng trnh 32.47 thng th qu phc tp s dng trong h thng da trn nn vi iu khin, do vy, trong khi thchin phng trnh 32.48 c s dng nhiu hn.

V d thit k

Xt vic thit k mt b iu khin m n gin cho h thng bnh ti. Thi gian ti l mt hm ca nhit v m.Bn hm lin thuc c s dng cho nhit , ba cho m v ba cho thi gian ti nh minh ha trn hnh 32.23. S dngtrc gic, bng m c th c trnh by nh trn hnh 32.24.

Gi s nhit l 600F v m l 70%. S dng hm lin thuc cho nhit v m, cc bin m sau y c c

cho nhit : [0, 0.2, 0.5, 0] v cho m: [0, 0.4, 0.6]. S dng hot ng min, bng m by gi c th c in vo vibin m tm thi, nh trn hnh 32.24 (b). Ch rng ch c bn c gi tr khc khng. S dng lut m, nh trn hnh32.24 (a), hot ng max c th c p dng thu c cc bin u ra m: short o 1 = max{0, 0, 0.2, 0.5, 0} = 0.5,medium o2 = max{0, 0, 0.2,0.4, 0} = 0.4, long o3 = max{0,0} = 0.

HNH 32.33 Hm lin thuc cho v d c miu t: (a) v (b) hm lin thuc cho bin u vo, (c) v (d) l hai hm linthuc dng cho bin u ra

16


17/20


HNH 32.24 Bng m: (a) cc lut m cho v d thit k,(b) cc bin m tm thi cho v d thit k

S dng phng trnh 32.47 v hnh 32.23 (c), xc nh c thi gian ti l 28 giy . Khi mt phng php n ginha c s dng vi phng trnh 32.46 v hnh 32.23 (d), th thi gian ti l 27 giy.

32.8 Thut ton di truyn

S thnh cng ca mng n ron nhn to khuyn khch cc nh nghin cu nghin cu cc mu khc trong t nhin.Kh nng ca di truyn thng qua s tin ha c th to ra nhng b my phc tp nh con ngi. Thut ton di truyn hccho php qu trnh tin ha nh trong t nhin tm ra gii php tt hn cho cc vn phc tp. C s cho thut ton ditruyn hc c a ra bi Holland (1975) v Goldberg (1989). Sau khi cho gi tr ban u, la chn cc bc, ti to vimt s giao nhau v s bin i c lp li cho mi th h. Trong th tc ny, mt chui cc k hiu c hiu nh lnhng nhim sc th, c c lng c mt gii php tt hn. Thut ton di truyn hc bt u vi vic m ha v chocc gi tr ban u. Tt c cc bc quan trng ca thut ton di truyn hc s c gii thch s dng mt v d n gin tm ra cc i ca hm (sin2(x)-0.5*x)2 vi di ca x t 0 n 1.6. Ch rng trong di ny, hm c gi tr cc i ton ccti x = 1.309, v cc i cc b ti x = 0.262.

M ha v gi tr ban u

Th nht, bin x phi c miu t nh mt chui cc k hiu. Vi chui di hn, qu trnh thng hi t nhanh hn, dovy cc k hiu t hn cho mt di cc chui c s dng s em li kt qu tt hn. Mc d chui ny c th l mt dycc k hiu no , k hiu nh phn 0 v 1 thng c s dng. Trong v d ny, 6 s nh phn c s dng cho vic mha, c gi tr trong h thp phn l 40x. Qu trnh x l bt u vi mt th h ngu nhin ca mu ban u c cho trongbng 32.1.

La chn v ti to

La chn nhng thnh vin tt nht ca mu l mt bc quan trng trong thut ton di truyn hc. Nhiu phng phpkhc nhau c th c s dng xp hng c th. Trong v d ny, hm xp hng c cho. Hng cao nht c s thnh vin6, v hng thp nht c s thnh vin 3. Nhng thnh vin vi hng cao hn s c c hi hn ti to. Xc sut ti to cho

mi thnh vin c th t c nh mt t l ca tng tt c cc gi tr hm i tng. T l ny c th hin ct ngoicng ca bng 32.1. Ch rng s dng phng php ny, hm i tng ca chng ta phi lun l dng. Nu khng,cn a vo trc mt s chun ha thch hp.

BNG 32.1 Mu ban u

S chui Chui Gi tr hthp phn

Gi tr bin Gi tr hm T l

1

2

3

4

56

7

101101

101000

010100

100101

001010110001

100111

45

40

20

37

1049

39

1.125

1.000

0.500

0.925

0.2501.225

0.975

0.0633

0.0433

0.0004

0.0307

0.00410.0743

0.0390

0.2465

0.1686

0.0016

0.1197

0.01580.2895

0.1521

17


18/20

S tay C in t

8

Total

000100 4 0.100 0.0016

0.2568

0.0062

1.0000

Ti to

Nhng s trong ct ngoi cng ca bng 32.1 th hin xc sut ti to. Do vy, hu nh s thnh vin 3 v 8 s khngc ti to, s 1 v 6 c th c hai hoc nhiu hn hai bn sao. S dng qu trnh ti to ngu nhin, mu sau y c t

chc thnh cp, c th c sinh ra:101101 45 110001 49 100101 37 110001 49

100111 39 101101 45 110001 49 101000 40

Nu kch c ca mu t mt th h ny ti mt th h khc l ging nhau, hai cha m s sinh ra hai ngi con. Bng cchkt hp hai chui, hai chui khc s c sinh ra. Cch n gin thc hin iu ny l tch lm hai phn mi chui cha mv trao i nhng chui con gia cha m. V d, t chui cha m 010100 v 100111, nhng chui con sau y s c sinhra: 010111 v 100100. Qu trnh ny c hiu nh l s giao nhau. Nhng chui con sinh ra l

101111 47 110101 49 100001 33 110000 48

100101 37 101001 41 110101 53 101001 41

Trng hp tng qut, cc chui khng cn phi tch lm i. N thng l nu ch la chn nhng bit c trao igia cha m v quan trng l v tr cc bit khng b thay i.

S bin i

Trong qu trnh tin ha v ti to s lm tng thm s bin i. Hn na, tnh cht tha k t cha m s dn n kt qul sinh ra mt s tnh cht ngu nhin mi. Qu trnh ny c hiu nh l s tin ha. Trong hu ht trng hp, s tin hasinh ra nhng con c th hng thp hn, v b loi tr trong qu trnh ti to. Tuy nhin, thnh thong s ti to c th cgii thiu mt cch c th tt hn vi mt tnh cht mi. iu ny bo v qu trnh ti to khi s thoi ha. Trong thut tondi truyn, s bin i thng ng vai tr th yu. i vi mc bin i cao, qu trnh ging vi th h mu ngu nhinv nh l mt thut ton tm kim rt khng hiu qu. T l bin i thng c gi s ti mc nh hn 1%. Trong v dny, s bin i tng ng vi s thay i bit ngu nhin ca mu cho. Trong trng hp n gin, vi nhng chuingn v mu nh, v vi t l bin i in hnh l 0.1%, nhng mu gi nguyn hu nh khng b thay i bi qu trnhbin i. Th h th hai cho v d ny c th hin trong bng 32.2.

BNG 32.2 Mu ca th h th hai

S chui Chui Gi tr hthp phn

Gi tr bin Gi tr hm T l

1

2

3

4

5

6

78

Total

010111

100100

110101

010001

100001

110101

110000101001

47

37

53

41

33

53

4841

1.175

0.925

1.325

1.025

0.825

1.325

1.2001.025

0.0696

0.0307

0.0774

0.0475

0.0161

0.0774

0.07220.0475

0.4387

0.1587

0.0701

0.1766

0.1084

0.0368

0.1766

0.16460.1084

1.0000

Ch rng hai thnh vin vi hng cao nht xc nh ca th h th hai rt gn vi nghim

x = 1.309. Cc mu c la chn ngu nhin cho th h th ba l

010111 47 110101 53 110000 48 101001 41

110101 53 110000 48 101001 41 110101 53

Ci to ra nhng con nh sau:

010101 21 110000 48 110001 49 101101 45110111 55 110101 53 101000 40 110001 49

18


19/20


Kt qu tt nht trong mu th ba nh trong mu th hai. Bng cch kim tra cn thn tt c nhng chui t th h th haiv th ba, c th kt lun rng s dng s giao nhau, cc chui lun c tch lm i, gii php tt nht 110100 52s khng bao gi t c, ng quan tm nht l c bao nhiu th h c to ra. y l l do khng c mu no trong thh th hai c chui con t ti 100. i vi s giao nhau, kt qu tt nht c th ch t c do qu trnh bin i, v yucu trong nhiu th h. Kt qu tt nht trong th h tng lai cng c th t c khi chui c tch ra ti cc v tr ngunhin. Mt gii php c th khc ch l nhng bit c la chn ngu nhin, c trao i gia cha m. Thut ton di truynhc rt nhanh, n dn n mt gii php tt trong mt vi th h. Gii php ny thng gn ging vi cc i ton cc,nhng khng tt nht.

nh ngha cc thut ng

Backpropagation: K thut hun luyn cho mng n ron nhiu lp.

Bipolar neuron:N ron vi u ra gia -1 v +1.

Feedforward network: Mng truyn thng (khng phn hi)

Perceptron: Mng perceptron

Recurrent network: Mng hi qui (c phn hi).

Supervised learning: Th tc hc khi u ra mong mun bit (hc gim st).

Unipolar neuron:N ron vi tn hiu gia 0 v +1.

Unsupervised learning: Th tc hc khi u ra mong mun cha bit (hc khng gim st).

Ti liu tham kho

[1] Fahlman, S.E. 1988. Faster-learning variations on backpropagation: An empirical study. Proceedings of theConnectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds., MorganKaufmann, San Mateo, CA.

[2] Fahlman, S.E. and Lebiere, C. 1990. The cascade correlation learning architecture.Adv. Ner. Inf. Proc. Syst., 2,D.S. Touretzky, ed., pp. 524532. Morgan Kaufmann, Los Altos, CA.

[3] Goldberg, D.E. 1989. Genetic Algorithm in Search, Optimization and Machine Learning. AddisonWesley, Reading,MA.

[4] Grossberg, S. 1969. Embedding fields: a theory of learning with physiological implications. Journal ofMathematical Psychology6:209239.

[5] Hebb, D.O. 1949. The Organization of Behavior, a Neuropsychological Theory. John Wiley, New York.[6] Hecht-Nielsen, R. 1987. Counterpropagation networks.Appl. Opt. 26(23):49794984.

[7] Hecht-Nielsen, R. 1988. Applications of counterpropagation networks. Neural Networks 1:131139.

[8] Holland, J.H. 1975.Adaptation in Natural and Artificial Systems. Univ. of Michigan Press, Ann Arbor, MI.

[9] Hopfield, J.J. 1982. Neural networks and physical systems with emergent collective computation abilities.Proceedings of the National Academy of Science 79:25542558.

[10] Hopfield, J.J. 1984. Neurons with graded response have collective computational properties like those of two-stateneurons. Proceedings of the National Academy of Science 81:30883092.

[11] Kohonen, T. 1988. The neural phonetic typerater. IEEE Computer27(3):1122.

[12] Kohonen, T. 1990. The self-organized map. Proc. IEEE 78(9):14641480.

[13] Kosko, B. 1987. Adaptive bidirectional associative memories.App. Opt. 26:49474959.

[14] Kosko, B. 1988. Bidirectional associative memories. IEEE Trans. Sys. Man, Cyb. 18:4960.

[15] McCulloch, W.S. and Pitts., W.H., 1943. A logical calculus of the ideas imminent in nervous activity.Bull. Math.Biophys. 5:115133.

[16] Minsky, M. and Papert, S. 1969. Perceptrons. MIT Press, Cambridge, MA.

[17] Nilsson, N.J. 1965.Learning Machines: Foundations of Trainable Pattern Classifiers. McGraw-Hill, New York.

[18] Nguyen, D. and Widrow, B. 1990. Improving the learning speed of 2-layer neural networks, by choosing initial valuesof the adaptive weights. Proceedings of the International Joint Conference on Neural Networks (SanDiego), CA, June.

[19] Pao, Y.H. 1989.Adaptive Pattern Recognition and Neural Networks. AddisonWesley, Reading, MA.

[20] Rosenblatt, F. 1958. The perceptron: a probabilistic model for information storage and organization in the brain.Psych. Rev. 65:386408.

[21] Rumelhart, D.E., Hinton, G.E., and Williams, R.J. 1986. Learning internal representation by error propagation.Parallel Distributed Processing. Vol. 1, pp. 318362. MIT Press, Cambridge, MA.

19


20/20

S tay C in t

[22] Sejnowski, T.J. and Rosenberg, C.R. 1987. Parallel networks that learn to pronounce English text. ComplexSystems 1:145168.

[23] Specht, D.F. 1990. Probalistic neural networks. Neural Networks 3:109118.

[24] Specht, D.F. 1992. General regression neural network. IEEE Trans. Neural Networks 2:568576.

[25] Wasserman, P.D. 1989. Neural Computing Theory and Practice. Van Nostrand Reinhold, New York.

[26] Werbos, P., 1974. Beyond regression: new tools for prediction and analysis in behavioral sciences. Ph.D. diss.,Harvard University.

[27] Widrow, B. and Hoff, M.E. 1960. Adaptive switching circuits. 1960 IRE Western Electric Show and ConventionRecord, Part 4 (Aug. 23):96104.

[28] Widrow, B. 1962. Generalization and information storage in networks of adaline Neurons. In SelforganizingSystems, M.C. Jovitz, G.T. Jacobi, and G. Goldstein, eds., pp. 435461. Sparten Books,Washington, D.C.

[29] Wilamowski, M. and Torvik, L., 1993. Modification of gradient computation in the back-propagation algorithm.ANNIE93 -Artificial Neural Networks in Engineering. November 1417, 1993, St. Louis, Missou.; alsoin C.H Dagli, ed. 1993. Intelligent Engineering Systems Through Artificial Neural Networks Vol. 3,pp. 175180. ASME Press, New York.

[30] Zadeh, L.A. 1965. Fuzzy sets. Information and Control 8:338353.

[31] Zurada, J. 1992. Introduction to Artificial Neural Systems. West Publ.

20

sổ tay cdt chuong 32-noron va hthg mo

Documents