第5章 時系列データのモデリング, 補助情報を考慮したモデリング

Download 第5章 時系列データのモデリング, 補助情報を考慮したモデリング

Post on 15-Apr-2017

1.457 views

Category:

Science

0 download

Embed Size (px)

TRANSCRIPT

  • 5

    7

    @ksmzn

    :ALBERT

    December 17, 2015

    @ksmzn 5 December 17, 2015 1 / 39

  • Koshi @ksmzn Python

    @ksmzn 5 December 17, 2015 2 / 39

  • @ksmzn 5 December 17, 2015 3 / 39

  • 1 5.3.1

    2 5.3.2

    3 5.3.3

    4 DTM

    5 5.4

    6 References

    @ksmzn 5 December 17, 2015 4 / 39

  • 1 5.3.1

    2 5.3.2

    3 5.3.3

    4 DTM

    5 5.4

    6 References

    @ksmzn 5 December 17, 2015 5 / 39

  • k

    @ksmzn 5 December 17, 2015 6 / 39

  • DTM

    Dynamic Topic Model (Blei [2008]) LDA k (1:T )k

    (1:T )k =((1)k ,

    (2)k , . . . ,

    (T )k

    ) t t 1 (t)k N

    ((t1)k ,

    2I)

    (t)k

    @ksmzn 5 December 17, 2015 7 / 39

  • DTM

    @ksmzn 5 December 17, 2015 8 / 39

  • 1 5.3.1

    2 5.3.2

    3 5.3.3

    4 DTM

    5 5.4

    6 References

    @ksmzn 5 December 17, 2015 9 / 39

  • KL

    @ksmzn 5 December 17, 2015 10 / 39

  • y(t) N(x(t), 2I

    ), x(t) N

    (x(t1), 2I

    )x(t)

    @ksmzn 5 December 17, 2015 11 / 39

  • y(1:T ) x(t) (5.44)

    x(t) = E[x(t)|y(1:T )]

    y(1:T ) x(t) p

    (x(t)|y(1:T )

    )

    @ksmzn 5 December 17, 2015 12 / 39

  • p189-190

    p(x(t)|y(1:T )

    )=

    p (x(t+1)|x(t), 2)p(x(t+1)|y(1:t), 2, 2) p (x(t)|y(1:t), 2, 2)

    p(x(t+1)|y(1:T ), 2, 2

    )dx(t+1)

    p(x(t)|y(1:t), 2, 2

    )

    @ksmzn 5 December 17, 2015 13 / 39

  • p(x(t)|y(1:t), 2, 2

    ) N

    (x(t)|y(t), 2

    )p(x(t)|y(1:t1), 2, 2

    )

    p(x(t)|y(1:t1), 2, 2

    )

    p(x(t)|y(1:t), 2, 2

    )

    @ksmzn 5 December 17, 2015 14 / 39

  • p(x(t+1)|y(1:t), 2, 2

    )=

    p(x(t+1)|x(t), 2

    )p(x(t)|y(1:t), 2, 2

    )dx(t)

    p(x(t)|y(1:t), 2, 2

    )

    p(x(t+1)|y(1:t), 2, 2

    )

    @ksmzn 5 December 17, 2015 15 / 39

  • p(x(t)|y(1:t1), 2, 2

    )

    p(x(t)|y(1:t), 2, 2

    )

    p(x(t+1)|y(1:t), 2, 2

    )

    p(x(1)|y(0)

    )

    p(x(t)|y(1:t1)

    )

    p

    (x(t)|y(1:t)

    )

    @ksmzn 5 December 17, 2015 16 / 39

  • (5.46)

    p(x(t)|y(1:T )

    )=

    p (x(t+1)|x(t), 2)p(x(t+1)|y(1:t), 2, 2) p (x(t)|y(1:t), 2, 2)

    p(x(t+1)|y(1:T ), 2, 2

    )dx(t+1)

    (5.54)

    p(x(t+1)|y(1:t), 2, 2

    )

    @ksmzn 5 December 17, 2015 17 / 39

  • tp

    (x(T )|y(1:T ), 2, 2

    )

    p(x(t)|y(1:T ), 2, 2

    )

    @ksmzn 5 December 17, 2015 18 / 39

  • p(x(t)|y(1:T ), 2, 2

    )

    p191, p195p(x(t)|y(1:t), 2, 2

    )

    p(x(t)|y(1:T ), 2, 2

    )

    p(x(t)|y(1:t), 2, 2

    )= N

    (x(t)|m(t), (t)2I

    )p(x(t)|y(1:T ), 2, 2

    )= N

    (x(t)|m(t), (t)2I

    ) m(t)m(t) m(t+1)

    @ksmzn 5 December 17, 2015 19 / 39

  • 1 5.3.1

    2 5.3.2

    3 5.3.3

    4 DTM

    5 5.4

    6 References

    @ksmzn 5 December 17, 2015 20 / 39

  • DTM

    @ksmzn 5 December 17, 2015 21 / 39

  • LDA+

    k (1:T )k

    (t)k,v =exp

    (k,v

    (t))

    Vv=1 exp

    ((t)k,v

    ) , (t)k N ((t1)k , 2I)

    w(t)d,i Multi((t)d

    ), z(t)d,i Multi

    ((t)d

    ), (t)d Dir

    ((t)

    )

    @ksmzn 5 December 17, 2015 22 / 39

  • KL[q (z, ,) || p (z, , | w,,)]q (z, ,).

    log p (w | ,)

    F[q (z, ,)] q (z, ,).

    q (z, ,), q (z), q (),q ().

    @ksmzn 5 December 17, 2015 23 / 39

  • DTM

    q((1:T ), (1:T ), z(1:T )

    )=

    Kk=1

    q((1:T )k

    ) Tt=1

    M(t)d=1

    q((t)d

    )q(z(t)d

    )

    @ksmzn 5 December 17, 2015 24 / 39

  • q((1:T )k

    )

    (1:T )k

    (1:T )k @ksmzn 5 December 17, 2015 25 / 39

  • q((1:T )k

    )

    (1:T )k q((t)k

    )

    q((t)k |

    (1:T )k

    )= N

    ((t)k |m

    (t)k ,

    (t)2k I

    )(1:T )k

    @ksmzn 5 December 17, 2015 26 / 39

  • p207-208

    logp(w(1:T )

    ) F

    [q((1:T )

    )]+(q((t)d

    )q(z(t)d

    )

    ) L

    [q((1:T )

    )]+(q((t)d

    )q(z(t)d

    )

    )() L

    [q((1:T )

    )] q

    ((t)k |

    (1:T )k

    )

    q((t)d

    ), q

    (z(t)d

    )

    @ksmzn 5 December 17, 2015 27 / 39

  • 1 5.3.1

    2 5.3.2

    3 5.3.3

    4 DTM

    5 5.4

    6 References

    @ksmzn 5 December 17, 2015 28 / 39

  • DTMScience

    18811999 120 15955 20

    @ksmzn 5 December 17, 2015 29 / 39

  • Topic Model

    Web

    @ksmzn 5 December 17, 2015 30 / 39

  • DTM R gensimhttps:

    //radimrehurek.com/gensim/models/dtmmodel.html

    Sean M GerrishC++https://code.google.com/p/

    princeton-statistical-learning/downloads/detail?

    name=dtm_release-0.8.tgz

    berobero11Stanhttp:

    //heartruptcy.blog.fc2.com/blog-entry-138.html

    @ksmzn 5 December 17, 2015 31 / 39

    https://radimrehurek.com/gensim/models/dtmmodel.htmlhttps://radimrehurek.com/gensim/models/dtmmodel.htmlhttps://code.google.com/p/princeton-statistical-learning/downloads/detail?name=dtm_release-0.8.tgzhttps://code.google.com/p/princeton-statistical-learning/downloads/detail?name=dtm_release-0.8.tgzhttps://code.google.com/p/princeton-statistical-learning/downloads/detail?name=dtm_release-0.8.tgzhttp://heartruptcy.blog.fc2.com/blog-entry-138.htmlhttp://heartruptcy.blog.fc2.com/blog-entry-138.html

  • 1 5.3.1

    2 5.3.2

    3 5.3.3

    4 DTM

    5 5.4

    6 References

    @ksmzn 5 December 17, 2015 32 / 39

  • 1.

    2.

    @ksmzn 5 December 17, 2015 33 / 39

  • LDA-d

    d Dir ()

    xd =

    (xd,1, xd,2, . . . , xd,C

    ) C

    d axd = 1, xd = 0

    xdk

    @ksmzn 5 December 17, 2015 34 / 39

  • LDAMimno

    k xd f (xd) = Tk xd k

    Dirichletk = exp

    (Tk xd

    )

    Dirichletk N

    (0, 2I

    )d Dir

    (exp

    (T1 xd

    ), exp

    (T2 xd

    ), . . . , exp

    (Tk xd

    ))@ksmzn 5 December 17, 2015 35 / 39

  • zd,i

    k exp(Tk xd

    )

    z

    @ksmzn 5 December 17, 2015 36 / 39

  • @ksmzn 5 December 17, 2015 37 / 39

  • References

    [1] (2015) ()

    [2] Blei, D.M. and Lafferty, J.D. (2006) Dynamic Topic Models. Proceedings of the 23rdinternational Conference on Machine Learning. 113-120.

    [3] Mimno, D.M. and McCallum, A. (2008) Topic Models Conditioned on Arbitrary Features withDirichlet-multinomial Regression. in UAI. 411-418.

    [4] Topic Model - NAOKI ORIIS BLOGhttp://mrorii.github.io/blog/2013/12/27/

    analyzing-dazai-osamu-literature-using-topic-models/

    [5] Web - #kichi-memohttp://seikichi.hatenablog.com/entry/2013/04/29/013608

    @ksmzn 5 December 17, 2015 38 / 39

    http://mrorii.github.io/blog/2013/12/27/analyzing-dazai-osamu-literature-using-topic-models/http://mrorii.github.io/blog/2013/12/27/analyzing-dazai-osamu-literature-using-topic-models/http://seikichi.hatenablog.com/entry/2013/04/29/013608

  • .

    @ksmzn 5 December 17, 2015 39 / 39

    5.3.1 5.3.2 5.3.3 DTM 5.4 References