: 5 . . 5: 5 /: 0 /. 1 0 . 5

-5 : 5 CT T

? V]TWPOTL 6 W ]T R LM DP WT G T P[ T]b Q 6ST L

*(). (1 )- B6? *().

lixirong.net 1

? V]TWPOTL ?PP] L R LRP

• WLRP NL ]T T R T ] L ] WL]TNLVVb RP P[L]P L . ? . . ?.P ]P NP ]SL] OP N[TMP .5 P ]T]TP L O P P ] T L RT P TWLRP&

����������F O R VLbT R MPLNS

VTaT[ R& P] *

E]L]P Q ]SP 4[] WLRP 6L ]T T R

• : RVPf ES L O FPVV W OPVJHT bLV 6HBD*()-

• WLRP P N OT R Mb 6 L O P ]P NP OPN OT R Mb EF?

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

R ES [] FP[W ?PW [b EF? T]

VTaT[ R& P] +

7L]L T 5PN WT R L 5 ]]VP PNU 4RLT

• 7P PV WP ] Q TWLRP P ]P NP N [ [L

9VTNU[0U

. . 0.

.*()+ *(), *()- *().

9VTNU[+(U

?E6A6A

HT LV:P WP

*()/ d

)(U

)-(U

VTaT[ R& P] ,

A [ ? ]T P

• ?L b W [P 0: 0 L ]L]T ]SL 0 L ]L]T• / WTVVT ST] SP PL[NST R Q [ eO Rf ]SP 9VTNU[ PM T]P• ?L b T LV N NP ] NL MP PQQPN]T PVb [POTN]PO

• 6L P ]TVTcP T P N NP ] VP PV L ]L]T Q [ TWLRP NL ]T T R3

VTaT[ R& P] -

A [ B[ LV

• W [ T R EAF4 Mb N NP ] ML PO P ]P NP [P[L UT R

VTaT[ R& P] .

EP ]P NP DP DL UT R

• DP N W ]P P ]P NP N [P T ]P[W Q N NP ] OP]PN]T [P V]

PTRS] P ]EN [P Sb EP ] ) PTRS] N NP ]EN [P Sb EP ]

• Sb EP ]2 L NL OTOL]P P ]P NP [POTN]PO Mb ]SP EF? W OPV• P ]EN [P Sb EP ] 2 T T]TLV N [P• N NP ]EN [P Sb EP ] 2 L P[LRPO N [P Q N NP ] Q O T ]SP Sb EP ]

VTaT[ R& P] /

EP ]P NP DP DL UT R2 DPLV 8aLW VP

VTaT[ R& P] 0

EP ]P NP DP DL UT R2 DPLV 8aLW VP

VTaT[ R& P] 1

? OPV Q [ 6 NP ] 7P]PN]T

• PTRSM [ ]T R PTH ]P J T F??*((1

• 6 ] TWLRP PTRSM [ VLMPVPO T]S L PNTQTN N NP ]

T P] LV& E NTLVTcT R ]SPPWL ]TN RL 2 4 N W L[L]T P[ Pb TWLRP ]LR L TR WP ]

[PQT PWP ] L O [P][TP LV 46?6 W ]T R E [ Pb *().

VTaT[ R& P] )(

? OPV Q [ 6 NP ] 7P]PN]T

• TP[L[NSTNLV EPWL ]TN 8WMPOOT R TP[E8 J T E : D*()-

• 6 W ]P TWLRP N NP ] [PVP L NP T L I [O*HPN LNPJ?TU V BE*()+

• 9L ]P[ ]SL PTH ]P

VTaT[ R& P] ))

8 LV L]T

• WLRP6 89 *()- WLRP EP ]P NP :P P[L]T FL UJ:TVMP[] 6 89*()-

• 4 OP P] Q *U TWLRP T]S WL LVVb [T]]P NL ]T• - ] -) NL ]T P[ TWLRP

• BL[]TNT L ] P[P L UPO ] [POTN] NL ]T Q [ PM TWLRP• FSP [RL TcP[ PVPN] L M P] Q + (/( TWLRP STNS P O ] U Q [ ]P ]• W TMVP ] WL LVVb ] P Q [ gR Oh P[Q [WL NP

• 8 LV L]T N[T]P[T• ?8F8AD

VTaT[ R& P] )*

DP V]

• BP[Q [WL NP Q [ b ]PW OP[ L[TPO P]]T R

ML PVT P à

VTaT[ R& P] )+

DP V]

• 6 W L[T R T]S ]SP[ MWT T T WLRP6 89 *()-• A [ [ L[P S T N V [ ML[

VTaT[ R& P] ),

8a]P T ] ]SP TOP O WLT

• 6 NP ] ML PO P ]P NP [P[L UT R LV [U Q [ TOP NL ]T T R

7 R P] LV& 8L[Vb PWMPOOT R L O VL]P [P[L UT R Q [ TOP NL ]T T R 46? ? V]TWPOTL *().

VTaT[ R& P] )-

6 NV T

• 6 NP ] ML PO P ]P NP [P[L UT R T SPV Q V• :T P [ P[ N NP ] OP]PN]T NLM VL[b W OPV d

• W [ P TWLRP NL ]T T R T]S Pa][L N ] Q WL LV L ]L]T• PL[ T R Q[ W 9VTNU[ TWLRP

• IT T R P ][b T WLRP6 89 *()-

VTaT[ R& P] ).

7PW 2 WLRP NL ]T T R T 6ST P P

5 ? 0: 5 0.

VTaT[ R& P] )/