Download - 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008
![Page 1: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/1.jpg)
02/12/2008 1
a tutorial on
Markov Chain Monte Carlo (MCMC)
Dima Damen
Maths ClubDecember 2nd 2008
![Page 2: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/2.jpg)
02/12/2008
2/33Markov Chain Monte Carlo – a tutorial
Plan
Monte Carlo IntegrationMarkov ChainsMarkov Chain Monte Carlo (MCMC)Metropolis-Hastings AlgorithmGibbs SamplingReversible Jump MCMC (RJMCMC)Applications
MAP estimation – Simulated MCMC
![Page 3: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/3.jpg)
02/12/2008
3/33Markov Chain Monte Carlo – a tutorial
Monte Carlo Integration
Stan Ulam (1946) [1]
![Page 4: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/4.jpg)
02/12/2008
4/33Markov Chain Monte Carlo – a tutorial
Monte Carlo Integration Any distribution π can be approximated by a set of
samples of size n where the distribution of the samples π ⋆
Monte Carlo simulation assumes independent and identically-distributed (i.i.d.) samples.
n
ttxfn
xfE1
)(1
)]([
![Page 5: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/5.jpg)
02/12/2008
5/33Markov Chain Monte Carlo – a tutorial
Markov Chains
Andrey Markov (1885)
![Page 6: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/6.jpg)
02/12/2008
6/33Markov Chain Monte Carlo – a tutorial
Markov Chains
To define a Markov chain you needSet of states (D) / domain (C)Transition matrix (D) / transition probability (C)Length of the Markov chain – nStarting state (s0)
![Page 7: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/7.jpg)
02/12/2008
7/33Markov Chain Monte Carlo – a tutorial
Markov Chains
A B
C D
0.4
0.3
0.3 0.1
0.5
0.20.2
0.3
0.4
0.5
0.5
0.3
02.05.03.0
5.05.000
2.04.01.03.0
03.04.03.0
C C D B B A C D A
![Page 8: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/8.jpg)
02/12/2008
8/33Markov Chain Monte Carlo – a tutorial
Markov Chain - proof
A right stochastic matrix A is a matrix where A(i, j) ≥ 0 and the sum of each row = 1
Exists but not guaranteed to be unique.if the Markov chain is irreducible and
aperiodic, the stationary distribution is unique
MatlabMatlab
![Page 9: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/9.jpg)
02/12/2008
9/33Markov Chain Monte Carlo – a tutorial
Markov Chain Monte Carlo (MCMC)
Used for realistic statistical modelling 1953 – Metropolis1970 – Hastings et. al.
![Page 10: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/10.jpg)
02/12/2008
10/33Markov Chain Monte Carlo – a tutorial
Markov Chain Monte Carlo (MCMC)
[2]
![Page 11: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/11.jpg)
02/12/2008
11/33Markov Chain Monte Carlo – a tutorial
Markov Chain Monte Carlo (MCMC)
[2]
![Page 12: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/12.jpg)
02/12/2008
12/33Markov Chain Monte Carlo – a tutorial
Markov Chain - proof
Detailed balance
then the invariant distribution is guaranteed to be unique and equals π.
proof[3]
![Page 13: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/13.jpg)
02/12/2008
13/33Markov Chain Monte Carlo – a tutorial
Markov Chain - proof
A B
0.6
0.4
Q(B|A) π(A)=Q(A|B)π(B)? (0.6) = ?? (0.4)
Q(A|B) = 3/2 Q(B|A)
A B
0.3
0.45
0.70.55
![Page 14: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/14.jpg)
02/12/2008
14/33Markov Chain Monte Carlo – a tutorial
Markov Chain Monte Carlo (MCMC)
For a selected proposal distribution Q(y|x), where , most likely Q will not satisfy the detailed balance for all (x, y) pairs. We might find that for some x and y choices
The process would then move from x to y too often and from y to x too rarely
![Page 15: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/15.jpg)
02/12/2008
15/33Markov Chain Monte Carlo – a tutorial
Markov Chain Monte Carlo (MCMC)
A convenient way to correct this condition is to reduce the number of moves from x to y by introducing an acceptance probability
[4]
![Page 16: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/16.jpg)
02/12/2008
16/33Markov Chain Monte Carlo – a tutorial
Markov Chain Monte Carlo (MCMC)
![Page 17: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/17.jpg)
02/12/2008
17/33Markov Chain Monte Carlo – a tutorial
Metropolis-Hastings algorithmAccepting the moves with a probability
guarantees convergence.But the performance can not be known in
advance.It might take too long to converge depending
on the choice of the transition matrix QA transition matrix where the majority of the
moves are rejected converges slower. The acceptance rate along the chain
is usually used to assess the performance
![Page 18: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/18.jpg)
02/12/2008
18/33Markov Chain Monte Carlo – a tutorial
The general MH algorithm
![Page 19: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/19.jpg)
02/12/2008
19/33Markov Chain Monte Carlo – a tutorial
Introduction to MCMC
MCMC – Markov Chain Monte CarloWhen?
You can’t sample from the distribution itselfCan evaluate it at any pointEx: Metropolis Algorithm
1
1
2
2
3
3
4
5
4 5 …1 4
![Page 20: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/20.jpg)
02/12/2008
20/33Markov Chain Monte Carlo – a tutorial
Metropolis-Hastings algorithm
When implementing MCMC, the most immediate issue is the choice of the proposal distribution Q.
Any proposal distribution will ultimately deliver (detailed balance), but the rate of convergence will depend crucially on the relationship between Q and π
![Page 21: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/21.jpg)
02/12/2008
21/33Markov Chain Monte Carlo – a tutorial
Metropolis-Hastings algorithm
Example
f(x) = 0.4 normpdf(x,2,0.5) + 0.6 betapdf (x,4,2)
???
proposal distribution
uniform distribution |y-x| <= 1
![Page 22: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/22.jpg)
02/12/2008
22/33Markov Chain Monte Carlo – a tutorial
Metropolis-Hastings algorithm
nmc = 100 nmc = 1,000
nmc = 10,000 nmc = 100,000
![Page 23: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/23.jpg)
02/12/2008
23/33Markov Chain Monte Carlo – a tutorial
Matlab Code
Examples…
![Page 24: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/24.jpg)
02/12/2008
24/33Markov Chain Monte Carlo – a tutorial
Matlab Code
![Page 25: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/25.jpg)
02/12/2008
25/33Markov Chain Monte Carlo – a tutorial
Metropolis-Hastings Algorithm
Burn-in time
Mixing time
Figure from [5]
![Page 26: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/26.jpg)
02/12/2008
26/33Markov Chain Monte Carlo – a tutorial
Running multiple chains
Assists convergenceCheck convergence by different starting points
run until they are indistinguishable.Two schools – single long chain, multiple
shorter chains
![Page 27: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/27.jpg)
02/12/2008
27/33Markov Chain Monte Carlo – a tutorial
Gibbs Sampling
Special case of MH algoα = 1 always (we accept all
movesDivide the space into a set of
dimensions
X = (X1, X2, X3, … , Xd)
Each scan i,
Xi = π (Xi | X≠i)
Figure from [1]
![Page 28: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/28.jpg)
02/12/2008
28/33Markov Chain Monte Carlo – a tutorial
Trans-dimensional MCMC
Choosing model size and parametersEx. # of Gaussians (k) and Gaussian parameters
(θ)Within model vs. across modelTrans-dimensional MCMCEx. RJMCMC (Green)
![Page 29: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/29.jpg)
02/12/2008
29/33Markov Chain Monte Carlo – a tutorial
Reversible Jump MCMC (RJMCMC)
Green (1995) [6]
joint distribution of model dimension and model parameters needs to be optimized to find the best pair of dimension and parameters that suits the observations.
Design moves for jumping between dimensions
Difficulty: designing moves
![Page 30: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/30.jpg)
02/12/2008
30/33Markov Chain Monte Carlo – a tutorial
Reversible Jump MCMC (RJMCMC)
![Page 31: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/31.jpg)
02/12/2008
31/33Markov Chain Monte Carlo – a tutorial
Application – MAP estimation
Maximum a Posteriori (MAP)Adding simulated annealing
![Page 32: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/32.jpg)
02/12/2008
32/33Markov Chain Monte Carlo – a tutorial
Application – MAP estimation
Figure from [1]
![Page 33: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/33.jpg)
02/12/2008 33
Thank you
![Page 34: 02/12/2008 1 a tutorial on Markov Chain Monte Carlo (MCMC) Dima Damen Maths Club December 2 nd 2008](https://reader035.vdocuments.mx/reader035/viewer/2022070308/551c1bc7550346ad4f8b58e1/html5/thumbnails/34.jpg)
02/12/2008
34/33Markov Chain Monte Carlo – a tutorial
References
[1] Andrieu, C., N. de Freitas, et al. (2003). An introduction to MCMC for machine learning. Machine Learning 50: 5-43
[2] Zhu, Dalleart and Tu (2005). Tutorial: Markov Chain Monte Carlo for Computer Vision. Int. Conf on Computer Vision (ICCV) http://civs.stat.ucla.edu/MCMC/MCMC_tutorial.htm
[3] Chib, S. and E. Greenberg (1995). "Understanding the Metropolis-Hastings Algorithm." The American Statistician 49(4): 327-335.
[4] Hastings, W. K. (1970). "Monte Carlo sampling methods using Markov chains and their applications." Biometrika 57(1): 97-109.
[5] Smith, K. (2007). Bayesian Methods for Visual Multi-object Tracking with Applications to Human Activity Recognition. Lausanne, Switzerland, Ecole Polytechnique Federale de Lausanne (EPFL). PhD: 272
[6] Green, P. (2003). Trans-dimensional Markov chain Monte Carlo. Highly structured stochastic systems. P. Green, N. Lid Hjort and S. Richardson. Oxford, Oxford University Press.