p14 hopfield neural networksyoga/courses/adapt/p14_hopfield... · 2020. 3. 9. · 4 the hopfield...
TRANSCRIPT
-
1
Hopfield Neural Networks
and
Their Applications
Dr. Yogananda Isukapalli
-
2
CONTENTS
q Introduction
q Hopfield Neural Networks
q Applications
-
3
The Hopfield Neural Network (HNN)
q Recurrent Neural Networkq One layer neural network with full connection
-
4
The Hopfield Neural Network
q Two aspects of Hopfield neural networks (HNN)§ Associative memory (content addressable memory)§ Optimization of energy function with quadratic form
q Associative memory ?
AddressDecoder Memory
x (address vector)
Address Addressable Memory
Memory
Content Addressable Memory
x(stimulus)
y(response)
-
5
Human perception with associate memory
-
6
Optimization
Find a minimum energy function
x
-
7
Applications
q DSP
q Communications
q Combinatorial optimization etc.,
-
8
Associative Memory (CAM)
q Types of CAM
- Matrix associative memory- Walsh associative memory- Network associative memory (Hopfield Neural Network)
-
9
Linear Associative Memory
q Memory Construction for associated pattern (xk, yk):Mk = xk* ykT
q Retrival:Mk*xk = < xkT *xk >* yk
q Overlay of patterns:
q Pattern Retrival:
åå==
==P
k
Tkk
P
kk yxMM
11.
åå=¹=
+==P
ikikii
P
kiki xMxMxMxM
11....
-
10
Walsh Associative Memory
q Walsh functions (Hadmard Matrix)
q Memory constructionM = Wk . xkT
q Retrival of Walsh Index
WkT . M . xk
q Pattern RetrivalWkT . M
úû
ùêë
é®ú
û
ùêë
é-
=22
222 11
1 1HHHH
H
-
11
Network Associative Memory
q Features of HNN:
Similar to Linear Associative MemoryNetwork Modeling
Asynchronous processing
Massive distributed and parallel processing
Simple architecture and analog VLSI implementation
-
12
The Hopfield Neural Network
q The Architecture of the Hopfield Neural Network
-
13
The Hopfield Neural Network
q Constructing T (Information Storage):
q Retrival:- The total input to the ith neuron:
- Updating
q Energy function of HNN:
q The change DE in E due to changing the state of ith neuronby Vi is
å=
-=P
k
Tkk pIVVT1
)()( .
åå¹
+=ji
ijiji IVTu
iij
ijiji
iij
ijiji
UIVTVV
UIVTVV
>+®
-
14
The Continuous Hopfield Neural Network
q Energy function
å òååå -¹
÷÷ø
öççè
æ+--=
i
V
iii
iiji
jiij
i
dVvgR
VIVVTE0
1 )(121
-
15
The Continuous Type Network
q Energy function of the Hopfield Network
q Equation of the Motion of the Network
q Input-output relationship
q Stability of the system
å òååå -¹
÷÷ø
öççè
æ+--=
i
V
iii
iiji
jiij
i
dVvgR
VIVVTE0
1 )(121
ij
jijii IVTRu
dtdu
++-= å
)/exp(11
)(Ru
ugi
i -+-=
21 ).('. ÷
øö
çè涶
-=¶¶ -
tVVgC
tE i
iii
-
16
The Continuous Type Network
-
17
Application 1 (A/D Converter)
q 4 bit A/D Converter:
q Energy function for the A/D converter:
q Energy function for Hopfield network:
xVi
ii Ȍ
=
3
02.
åå==
--÷ø
öçè
æ-=
3
0
223
0)]1.(.[)2(2.
iii
i
i
ii VVVxE
åå å=
-
= =¹
+ +----=3
0
)12(3
0
3
0).22()2(
21
ii
ii
j jiji
ji VxVVE
-
18
Application 1 (A/D Converter)
åå å=
-
= =¹
+ +----=3
0
)12(3
0
3
0).22()2(
21
ii
ii
j jiji
ji VxVVE
-
19
Application 2 (Multiuser Detection)
q Problem definition:
Recover binary information sent from multiple transmitters
q The receiver observes
],[ ),()().()(1
sss
K
kskk TjTjTttnjTtsjbtr +Î+-=å
=
s
)()().()(1
tnjTtsjbtrK
kskk s+-=å
=
-
20
q Maximum likelihood detection:- Maximize likelihood probability:
- Further developed as follows:
Application 2 (Multiuser Detection)
ò å
åò
úû
ùêë
é-Î
=
=
=Î
W
T K
kkk
ii
T
bb
dttsbtr
dttsb(r(t)-(b)where
etrP
0
2
1
max
{-1,1} b
2
0
)(
)()(b̂
))(.Ω
]|)([
arg K
òò ==
-ÎÎ
ss T
jiij
T
kk
TT
dttstsHdttstrywhere
Hbbby
00
max
{-1,1} b
)().( and )().(
2 b̂ arg K
-
21
-
22
Application 3 (Object Recognition)
q Problem definition:
Find out corresponding points between those in two objects ….
q An objective function for pattern recognition:
q Energy function for Hopfield network:
)(22 åååååååååå ¹¹
++-=k i ij
jkiki k kl
iliki j k l
jlikijkl VVVVqVVCAE
klijklijijklijkl
i kikik
i j k ljlikijkl
qqACTwhere
VIVVTE
dddd .2)( 21
++-=
+-= åååååå
-
23
Application 3 (Object Recognition)
-
24
Application 3 (Object Recognition)
-
25
Application 3 (Object Recognition)
-
26
Application 3 (Object Recognition)