mcmc estimation march30 2009

8/2/2019 MCMC Estimation March30 2009

1/19

1

MCMC Estimation

MCMC = Markov chain Monte Carloan alternative approach to estimating models


2/19

2

What is the big deal about Markov chain Monte Carlo methods?

While MCMC methods are not new, recent advances in algorithms using thesemethods have led to a bit of a revolution in statistics. This revolution is typicallyseen as a "Bayesian" revolution because of the fact that the MCMC methods havebeen put to work by relying on Bayes theorem. It turns out that combining theuse of Bayes theorem with MCMC sampling permits an extremely flexibleframework for data analysis. However, it is important for us to keep in mind that

a Bayesian approach to statistics and MCMC are separate things, not one and thesame.

For the past several years, MCMC estimation has given those adopting theBayesian philosophical perspective a large advantage in modeling flexibility overthose using other statistical approaches (frequentists and likelihoodists). Very

recently, it has become clear that the MCMC-Bayesian machinery can be used toobtain likelihood estimates, essentially meaning that one doesnt have to adopt aBayesian philosophical perspective to use MCMC methods. The good news is thatthere are a lot of new capabilities for analyzing data. The bad news is that thereare a lot of disparate perspectives in the literature (e.g., people attributing themerits of MCMC as being inherent advantages of the Bayesian perspective, plus

different schools of Bayesian analysis).


3/19

3

Bayesian Fundamentals

P(M|D) =P(D|M) P(M)

P(D)

where:

P(M|D) = the probability of a model/parameter value given the data

P(D|M) = the probability (likelihood) of the data given the model

P(M) = the prior probability of the model/parameter value givenprevious information

P(D) = the probability of observing these data given the

data-generating mechanism

1. Bayes Theorem


4/19

4

Bayesian Fundamentals (cont.)

2. The context of the Bayesian approach is to reduceuncertainty through the acquisition of new data.

3. What about that prior?

a. When our interest is predicting the next event, theprior information may be very helpful.

b. When our interest is in analyzing data, we usuallytry to use uninformative priors.

c. When we know something, like percentage datadon't go beyond values of 0 and 100, that can beuseful prior information to include.

d. The biggest worry about priors is that they mayhave unknown influences in some cases.


5/19

5

Bayesian Fundamentals (cont.)

4. Bayesian estimation is now frequently conducted usingMarkov Chain Monte Carlo (MCMC) methods. Suchmethods are like a kind of bootstrapping that estimatesthe shape of the posterior distribution.

5. MCMC methods can also be used to obtain likelihoods;remember,

posterior = likelihood * prior.

By data cloning as described in Lele* et al. (2007), it ispossible to obtain pure likelihood estimates using MCMC.

*Lele, Dennis, and Lutscher (2007) Data cloning: easy maximum likelihoodestimation for complex ecological models using Bayesian Markov chain

Monte Carlo methods. Ecology Letters 10:551-563.


6/19

6

MCMC Estimation in Amos

The next few slides give a few screen shots ofBayesian/MCMC estimation in Amos. I highly recommendthe brief video developed by Jim Arbuckle that can be foundat www.amosdevelopment.com/site_map.htm. Just go tothis site and look under videos for Bayesian Estimation:Intro.
http://www.amosdevelopment.com/site_map.htmhttp://www.amosdevelopment.com/site_map.htmhttp://www.amosdevelopment.com/site_map.htm


7/19


8/19

8

Illustration (cont.)

frown means not yet

converged


9/19

9


smile means program

converged; once you

have converted, you

can pause simulation

none of the 95% credible

intervals include the value

of 0. This indicates that weare 95% sure that the true

values of the parameters

fall within the CIs and are

nonzero.

point estimates


10/19

10


Some measures of model fit.

Posterior predictive p values providesome information on overall model

fit to data, with values closer to 0.50being better than ones larger orsmaller.

DIC values for different models canbe compared in a fashion similar to

the use of AIC or BIC.

Discussions of model comparison formodels using MCMC will bediscussed in a separate module.


11/19

11


shape of the prior for the parameter for thepath from cover to richness.

right-click on

parameter row

to select either

prior or posterior

for viewing


12/19

12


shape of the posterior for the parameter for the

path from cover to richness.

S.D. is the standard deviation

of the parameter.

S.E. is the precision of the

MCMC estimate determined

by how long you let the

process run, not the std. error!

there are important options

you can select down here,

like viewing the trace,

autocorrelation, or the first

and last half estimates.


13/19

13


shape of the trace for the parameter for the


The trace is our evaluation ofhow stable our estimate wasduring the analysis. Believe itor not, this is how you wantthe trace to look! It should notbe making long-term

directional changes.


14/19

14


shape of the autocorrelation curve for the parameter for the


The autocorrelation curvemeasures the asymptoticdecline to independence forthe solution values. You wantit to level off, as it has donehere.


15/19

15

Standardized Coefficients

To get standardized coefficients, including a full set of moments plus theirposteriors, you need to select "Analysis Properties", the "Output" tab, and thenplace a check mark in front of both "Standardized Estimates" and "Indirect,direct & total effects". If you don't ask for "Indirect, direct, and totol effects",you will not actually get the standardized estimates.

Then, when you have convergence from your MCMC run, go the the "View"dropdown and select, "Additional Estimands". You will probably have to grab anddrag the upper boundary of the subwindows on the left to get to see everythingproduced, but there should be a column of choices for you to view (shown onnext slide).

For more information about standardized coefficients in SEM, see, forexample,Grace, J.B. and K.A. Bollen. 2005. Interpreting the results from multipleregression and structural equation models. Bulletin of the Ecological Societyof America. 86:283-295.


16/19

16

Standardized Coefficients (cont.)

here you can choose various options

here you can see the results


17/19

17

Calculating R2 Values

Amos does not give the R2 values for response variables when the MCMC methodis used for estimation. Some statisticians tend to shy away from making a bigdeal about R2 values because they are properties of the sample rather than ofthe population. However, other statisticians and most subject-area scientists areusually quite interested in standardized parameters such as standardizedcoefficients and R2 values, which measure the "strength of relationships". On thenext slide I show one way to calculate R2 from the MCMC output. The readershould note that R2 values from MCMC analyses are (in my personal view)sometimes problematic in that they are noticably lower than a likelihood

estimation process would produce. I intend a module on this advanced topic atsome point.


18/19

18

Calculating R2 Values

R2 = 1 (e1/variance of salt_log)

We need the implied variance of response variables to calculate R2. To get impliedvariances in Amos, you can select that choice in the Output tab of the Analysis Propertieswindow. With the MCMC procedure, you have to request Additional Estimands from theView dropdown after the solution has converged. For this example, we get an estimate ofthe implied covariance of salt_log of 0.119. So, R2 = 1-(0.034/0.119) = 0.714. This

compares to the ML estimated R2

of 0.728. Again, I will have more to say in a latermodule about variances and errors estimated using MCMC.

error variance for response variable


19/19

19

Final Bit

Amos makes Bayesian estimation (very!) easy. Amos can do a

great deal more than what I have illustrated, like estimatecustom parameters (like the differences between values).Unfortunately, Amos cannot do all the kinds of things that canbe done in lower level languages like winBUGS or R. This may

change before too long (James Arbuckle, developer of Amos, isnot saying at the moment). For now, tapping the full potentialof MCMC methods requires the use of another softwarepackage, winBUGS (or some other package like R). I will bedeveloping separate modules on SEM using winBUGS in thenear future for those who want to use more complex models(and are willing to invest considerably more time).

mcmc estimation march30 2009

Documents