bishop latent erice 99

33
LATENT VARIABLE MODELS CHRISTOPHER M. BISHOP Microsoft Research 7 J. J. Thomson Avenue, Cambridge CB3 0FB, U.K. Published in Learning in Graphic al Models, M. I. Jordan (Ed.), MIT Press (1999), 371–403. Abstract. A powerful approach to probabilistic modelling involves sup- plementing a set of observed variables with additional latent, or hidden, variables. By dening a joint distribution over visible and latent variables, the corresponding distribution of the observed variables is then obtained by marginalization. This allows relatively complex distributions to be ex- pressed in terms of more tractable joint distributions over the expanded variable space. One well-known example of a hidden variable model is the mixture distribution in which the hidden variable is the discrete component label. In the case of continuous latent variables we obtain models such as factor analysis. The structure of such probabilistic models can be made particularly transparent by giving them a graphical representation, usually in terms of a directed acyclic graph, or Bayesian network. In this chapter we provide an overview of latent variable models for representing continuous variables. We show how a particular form of linear latent variable model can be used to provide a probabilistic formulation of the well-known tech- nique of principal components analysis (PCA). By extending this technique to mixtures, and hierarchical mixtures, of probabilistic PCA models we are led to a powerful interactive algorithm for data visualization. We also show how the probabilistic PCA approach can be generalized to non-linear latent variable models leading to the Generative Topographic Mapping algorithm (GTM). Finally, we show how GTM can itself be extended to model tem- poral data. 371

Upload: walternovillo3142

Post on 03-Apr-2018

230 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 1/33

Page 2: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 2/33

Page 3: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 3/33

Page 4: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 4/33

Page 5: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 5/33

Page 6: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 6/33

Page 7: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 7/33

Page 8: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 8/33

Page 9: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 9/33

Page 10: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 10/33

Page 11: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 11/33

Page 12: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 12/33

Page 13: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 13/33

Page 14: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 14/33

Page 15: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 15/33

Page 16: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 16/33

Page 17: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 17/33

Page 18: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 18/33

Page 19: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 19/33

Page 20: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 20/33

Page 21: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 21/33

Page 22: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 22/33

Page 23: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 23/33

Page 24: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 24/33

Page 25: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 25/33

Page 26: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 26/33

Page 27: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 27/33

Page 28: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 28/33

Page 29: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 29/33

Page 30: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 30/33

Page 31: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 31/33

Page 32: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 32/33

Page 33: Bishop Latent Erice 99

7/28/2019 Bishop Latent Erice 99

http://slidepdf.com/reader/full/bishop-latent-erice-99 33/33