# 第5章 時系列データのモデリング, 補助情報を考慮したモデリング

Post on 15-Apr-2017

1.457 views

Category:

## Science

Embed Size (px)

TRANSCRIPT

• 5

7

@ksmzn

:ALBERT

December 17, 2015

@ksmzn 5 December 17, 2015 1 / 39

• Koshi @ksmzn Python

@ksmzn 5 December 17, 2015 2 / 39

• @ksmzn 5 December 17, 2015 3 / 39

• 1 5.3.1

2 5.3.2

3 5.3.3

4 DTM

5 5.4

6 References

@ksmzn 5 December 17, 2015 4 / 39

• 1 5.3.1

2 5.3.2

3 5.3.3

4 DTM

5 5.4

6 References

@ksmzn 5 December 17, 2015 5 / 39

• k

@ksmzn 5 December 17, 2015 6 / 39

• DTM

Dynamic Topic Model (Blei [2008]) LDA k (1:T )k

(1:T )k =((1)k ,

(2)k , . . . ,

(T )k

) t t 1 (t)k N

((t1)k ,

2I)

(t)k

@ksmzn 5 December 17, 2015 7 / 39

• DTM

@ksmzn 5 December 17, 2015 8 / 39

• 1 5.3.1

2 5.3.2

3 5.3.3

4 DTM

5 5.4

6 References

@ksmzn 5 December 17, 2015 9 / 39

• KL

@ksmzn 5 December 17, 2015 10 / 39

• y(t) N(x(t), 2I

), x(t) N

(x(t1), 2I

)x(t)

@ksmzn 5 December 17, 2015 11 / 39

• y(1:T ) x(t) (5.44)

x(t) = E[x(t)|y(1:T )]

y(1:T ) x(t) p

(x(t)|y(1:T )

)

@ksmzn 5 December 17, 2015 12 / 39

• p189-190

p(x(t)|y(1:T )

)=

p (x(t+1)|x(t), 2)p(x(t+1)|y(1:t), 2, 2) p (x(t)|y(1:t), 2, 2)

p(x(t+1)|y(1:T ), 2, 2

)dx(t+1)

p(x(t)|y(1:t), 2, 2

)

@ksmzn 5 December 17, 2015 13 / 39

• p(x(t)|y(1:t), 2, 2

) N

(x(t)|y(t), 2

)p(x(t)|y(1:t1), 2, 2

)

p(x(t)|y(1:t1), 2, 2

)

p(x(t)|y(1:t), 2, 2

)

@ksmzn 5 December 17, 2015 14 / 39

• p(x(t+1)|y(1:t), 2, 2

)=

p(x(t+1)|x(t), 2

)p(x(t)|y(1:t), 2, 2

)dx(t)

p(x(t)|y(1:t), 2, 2

)

p(x(t+1)|y(1:t), 2, 2

)

@ksmzn 5 December 17, 2015 15 / 39

• p(x(t)|y(1:t1), 2, 2

)

p(x(t)|y(1:t), 2, 2

)

p(x(t+1)|y(1:t), 2, 2

)

p(x(1)|y(0)

)

p(x(t)|y(1:t1)

)

p

(x(t)|y(1:t)

)

@ksmzn 5 December 17, 2015 16 / 39

• (5.46)

p(x(t)|y(1:T )

)=

p (x(t+1)|x(t), 2)p(x(t+1)|y(1:t), 2, 2) p (x(t)|y(1:t), 2, 2)

p(x(t+1)|y(1:T ), 2, 2

)dx(t+1)

(5.54)

p(x(t+1)|y(1:t), 2, 2

)

@ksmzn 5 December 17, 2015 17 / 39

• tp

(x(T )|y(1:T ), 2, 2

)

p(x(t)|y(1:T ), 2, 2

)

@ksmzn 5 December 17, 2015 18 / 39

• p(x(t)|y(1:T ), 2, 2

)

p191, p195p(x(t)|y(1:t), 2, 2

)

p(x(t)|y(1:T ), 2, 2

)

p(x(t)|y(1:t), 2, 2

)= N

(x(t)|m(t), (t)2I

)p(x(t)|y(1:T ), 2, 2

)= N

(x(t)|m(t), (t)2I

) m(t)m(t) m(t+1)

@ksmzn 5 December 17, 2015 19 / 39

• 1 5.3.1

2 5.3.2

3 5.3.3

4 DTM

5 5.4

6 References

@ksmzn 5 December 17, 2015 20 / 39

• DTM

@ksmzn 5 December 17, 2015 21 / 39

• LDA+

k (1:T )k

(t)k,v =exp

(k,v

(t))

Vv=1 exp

((t)k,v

) , (t)k N ((t1)k , 2I)

w(t)d,i Multi((t)d

), z(t)d,i Multi

((t)d

), (t)d Dir

((t)

)

@ksmzn 5 December 17, 2015 22 / 39

• KL[q (z, ,) || p (z, , | w,,)]q (z, ,).

log p (w | ,)

F[q (z, ,)] q (z, ,).

q (z, ,), q (z), q (),q ().

@ksmzn 5 December 17, 2015 23 / 39

• DTM

q((1:T ), (1:T ), z(1:T )

)=

Kk=1

q((1:T )k

) Tt=1

M(t)d=1

q((t)d

)q(z(t)d

)

@ksmzn 5 December 17, 2015 24 / 39

• q((1:T )k

)

(1:T )k

(1:T )k @ksmzn 5 December 17, 2015 25 / 39

• q((1:T )k

)

(1:T )k q((t)k

)

q((t)k |

(1:T )k

)= N

((t)k |m

(t)k ,

(t)2k I

)(1:T )k

@ksmzn 5 December 17, 2015 26 / 39

• p207-208

logp(w(1:T )

) F

[q((1:T )

)]+(q((t)d

)q(z(t)d

)

) L

[q((1:T )

)]+(q((t)d

)q(z(t)d

)

)() L

[q((1:T )

)] q

((t)k |

(1:T )k

)

q((t)d

), q

(z(t)d

)

@ksmzn 5 December 17, 2015 27 / 39

• 1 5.3.1

2 5.3.2

3 5.3.3

4 DTM

5 5.4

6 References

@ksmzn 5 December 17, 2015 28 / 39

• DTMScience

18811999 120 15955 20

@ksmzn 5 December 17, 2015 29 / 39

• Topic Model

Web

@ksmzn 5 December 17, 2015 30 / 39

• DTM R gensimhttps:

name=dtm_release-0.8.tgz

berobero11Stanhttp:

//heartruptcy.blog.fc2.com/blog-entry-138.html

@ksmzn 5 December 17, 2015 31 / 39

• 1 5.3.1

2 5.3.2

3 5.3.3

4 DTM

5 5.4

6 References

@ksmzn 5 December 17, 2015 32 / 39

• 1.

2.

@ksmzn 5 December 17, 2015 33 / 39

• LDA-d

d Dir ()

xd =

(xd,1, xd,2, . . . , xd,C

) C

d axd = 1, xd = 0

xdk

@ksmzn 5 December 17, 2015 34 / 39

• LDAMimno

k xd f (xd) = Tk xd k

Dirichletk = exp

(Tk xd

)

Dirichletk N

(0, 2I

)d Dir

(exp

(T1 xd

), exp

(T2 xd

), . . . , exp

(Tk xd

))@ksmzn 5 December 17, 2015 35 / 39

• zd,i

k exp(Tk xd

)

z

@ksmzn 5 December 17, 2015 36 / 39

• @ksmzn 5 December 17, 2015 37 / 39

• References

[1] (2015) ()

[2] Blei, D.M. and Lafferty, J.D. (2006) Dynamic Topic Models. Proceedings of the 23rdinternational Conference on Machine Learning. 113-120.

[3] Mimno, D.M. and McCallum, A. (2008) Topic Models Conditioned on Arbitrary Features withDirichlet-multinomial Regression. in UAI. 411-418.

[4] Topic Model - NAOKI ORIIS BLOGhttp://mrorii.github.io/blog/2013/12/27/

analyzing-dazai-osamu-literature-using-topic-models/

[5] Web - #kichi-memohttp://seikichi.hatenablog.com/entry/2013/04/29/013608

@ksmzn 5 December 17, 2015 38 / 39

http://mrorii.github.io/blog/2013/12/27/analyzing-dazai-osamu-literature-using-topic-models/http://mrorii.github.io/blog/2013/12/27/analyzing-dazai-osamu-literature-using-topic-models/http://seikichi.hatenablog.com/entry/2013/04/29/013608

• .

@ksmzn 5 December 17, 2015 39 / 39

5.3.1 5.3.2 5.3.3 DTM 5.4 References

Recommended