Upload File

lecture 14: approximate inference sampling methods · lecture 14: approximate inference sampling...

  • Home
  • Documents
  • Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms
34
CS839: Probabilistic Graphical Models Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1

Upload: others

Post on 06-Aug-2020

13 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

CS839:ProbabilisticGraphicalModels

Lecture14:ApproximateInferenceSamplingMethods

TheoRekatsinas

1

Page 2: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Approachestoinference

2

• Exactinferencealgorithms• Theeliminationalgorithm• Message-passingalgorithm(sum-product,beliefpropagation)• Junctiontreealgorithm

• Approximateinferencetechniques• Variational algorithms

• Loopybeliefpropagation• Meanfieldapproximation

• Stochasticsimulation/samplingmethods• MarkovchainMonteCarlomethods

Page 3: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Howtorepresentajointdistribution?

3

• Closedformrepresentation

• Sample-basedrepresentationCollectsamplesX(m) ~P(x)ifwedrawalotofsampleswecanusethelawoflargenumberstogetthatEp(f(x))=Σmf(X(m))/|m|

Page 4: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

MonteCarloMethods

4

• Drawrandomsamplesfromthedesireddistribution• Yieldastochasticrepresentationofacomplexdistribution

• marginals andotherexpectationscanbeapproximatedusingsample-basedaverages• Ep(f(x))=Σmf(X(m))/|m|

• Asymptoticallyexactandeasytoapplytoarbitrarymodels• Challenges:

• howtodrawsamplesfromagivendist.(notalldistributionscanbetriviallysampled)?

• howtomakebetteruseofthesamples(notallsampleareuseful,orequallyuseful,seeanexamplelater)?

• howtoknowwe'vesampledenough?

Page 5: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

MonteCarloMethods

5

• DirectSampling• Wehaveseenit.• Verydifficulttopopulateahigh-dimensionalstatespace

• RejectionSampling• Createsampleslikedirectsampling,onlycountsampleswhichisconsistentwithgivenevidences.

• Likelihoodweighting,...• Samplevariablesandcalculateevidenceweight.Onlycreatethesampleswhichsupporttheevidences.

• MarkovchainMonteCarlo(MCMC)• Metropolis-Hasting• Gibbs

Page 6: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Rejectionsampling

6

• Supposewewishtosamplefromdist.Π(X)=Π'(X)/Z.• Π(X)isdifficulttosample,butΠ'(X)iseasytoevaluate• SamplefromasimplerdistributionQ(X)• Rejectionsampling

• Correctness:

• Pitfall:Wegainedasamplebutwhatdidwepay?

Page 7: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Unnormalized ImportanceSampling

7

• SupposesamplingfromP(·)ishard.• Supposewecansamplefroma"simpler"proposaldistributionQ(·)instead.• IfQdominatesP(i.e.,Q(x)>0wheneverP(x)>0),wecansamplefromQandreweight:

Page 8: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Normalizedimportancesampling

8

• SupposewecanonlyevaluateP’(x)=aP(x)

Page 9: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Weightedresampling

9

• Problemofimportancesampling:performancedependsonhowwellQmatchesP• IfP(x)f(x)isstronglyvaryingandhasasignificantproportionofitsmassconcentratedinasmallregion,rm willbedominatedbyafewsamples

• Solution:useaheavytailQandweightedresampling

Page 10: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

LimitationsofMonteCarlo

10

• Directsampling• Hardtogetrareeventsinhigh-dimensionalspaces• InfeasibleforMRFsunlessweknowthenormalizerZ

• Rejectionsampling,Importancesampling• WeneedagoodproposalQ(x)thatisnotverydifferentthanP(x)

• Howaboutweuseanadaptiveproposal?

Page 11: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

MarkovChainMonteCarlo

11

• MCMCalgorithmsfeatureadaptiveproposals• InsteadofQ(x’)useQ(x’|x)wherex’isthenewstatebeingsampledandxistheprevioussample• AsxchangesQ(x’|x)canalsochange

Page 12: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Metropolis-Hastings

12

• Drawasamplex’fromQ(x’|x)wherexistheprevioussample• Thenewsamplex’isacceptedorrejectedwithsomeprobabilityA(x’|x)

• Acceptanceprob:

• A(x’|x)islikearationofimportancesamplingweights• P(x’)/Q(x’|x)istheimportanceweightforx’,P(x)/Q(x|x’)istheimportanceweightforx• Wedividetheimportanceweightforx’bythatofx• NoticethatweonlyneedtocomputeP(x’)/P(x)ratherthanP(x’)orP(x)

• A(x’|x)ensuresthataftersufficientlymanydraws,oursamplescomefromthetruedistribution.

Page 13: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Metropolis-Hastings

13

Page 14: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ExampleofMH

14

• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)

Page 15: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ExampleofMH

15

• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)

Page 16: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ExampleofMH

16

• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)

Page 17: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ExampleofMH

17

• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)

Page 18: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ExampleofMH

18

• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)

Page 19: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ExampleofMH

19

• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)

Page 20: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ExampleofMH

20

• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)

Page 21: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ExampleofMH

21

• LetQ(x’|x)beaGuassian centeredonx• WearetryingtosamplefromabimodalP(x)

Page 22: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

SometheoreticalaspectsofMCMC

22

• TheMHalgorithmhasaburn-inperiod• InitialsamplesarenottrulyfromP

• WhyaretheMHsamplesguaranteedtobefromP(x)?• TheproposalQ(x’|x)keepschangingwiththevalueofx;howdoweknowthesampleswilleventuallycomefromP(x)?

• WhyMarkovChain?

Page 23: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

MarkovChains

23

• AMarkovChainisasequenceofrandomvariablesx1,x2,…,xN withtheMarkovProperty

• Therighthandsideisthetransitionkernel.Nextstatedependsonlyonprecedingstate

• Let’sassumethekernelisfixedwithtime.

Page 24: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

MCConcepts

24

Page 25: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

MCConcepts

25

• StationarydistributionsareofgreatimportanceinMCMC.Somenotions• Irreducible:anMCisirreducibleifyoucangetfromanystatextoanyotherstatex’withprobabilityx>0inafinitenumberofsteps• Aperiodic:anMCisaperiodicifyoucanreturntoanystatexatanytime• Ergodic(orregular):anMCisergodicifitisirreducibleandaperiodic

• Ergodicityisimportant:itimpliesyoucanreachthestationarydistributionnomattertheinitialdistribution.

Page 26: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

MCConcepts

26

• Reversible(detailedbalance):anMCisreversibleifthereexistsadistributionπ(x)suchthatthedetailedbalanceconditionholds

• ReversibleMCsalwayshaveastationarydistribution

Page 27: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

WhydoesMHwork?

27

• Wedrawasamplex’accordingtoQ(x’|x)andthenaccept/rejectaccordingtoA(x’|x).Hencethetransitionkernelis:

• WecanprovethatMHsatisfiesdetailedbalance.

Page 28: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

WhydoesMHwork?

28

• NowsupposeA(x’|x)<1andA(x|x’).=1.Wehave

• Thisisthedetailedbalancecondition:• TheMHalgorithmleadstoastationarydistributionP(x)• WedefinedP(x)tobethetruedistributionofx• Thus,MHeventuallyconvergestothetruedistribution

Page 29: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

GibbsSampling

29

• GibbsSamplingisanMCMCalgorithmthatsampleseachrandomvariableofagraphicalmodel,oneatatime

• GSisfairlyeasytoderiveformanygraphicalmodels

• GShasreasonablecomputationandmemoryrequirements(becausewesampleoner.v.atatime)

Page 30: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

GibbsSamplingAlgorithm

30

Page 31: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

GibbsSamplingExample

31

Page 32: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ParallelGibbsSampling

32

x

VariableTally

CompleteModelCopies

• RunGibbsindependentlyonfullcopiesofthesamemodel

• Feweriterationspercopy• Moresamplesmeansmore

accuratemarginals

Datatomaterializefactorgraph

RunsequentialGibbs

Page 33: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

ParallelGibbsSamplingVariableAssignments

ColoredModel

Datatomaterializefactorgraph

VariableTally• Computeak-coloringofthe

factorgraph• Sampleallvariableswithsame

colorinparallel• Loadbalancingisakey

challenge

t1

t1

t1

CoordinatedWorkers

Page 34: Lecture 14: Approximate Inference Sampling Methods · Lecture 14: Approximate Inference Sampling Methods Theo Rekatsinas 1. Approaches to inference 2 •Exact inference algorithms

Summary

34

• Samplingcanbeeasytoimplementbutwecangetpoorqualitysamples• Weneedagoodproposaldistribution

• MarkovChainMonteCarlomethodsuseadaptiveproposalsQ(x’|x)tosamplefromthetruedistributionP(x)

• Metropolis-HastingsallowsyoutospecifyanyproposalQ(x’|x)

• GibbssamplingsetstheproposalQ(x’|x)totheconditionalP(x’|x)• Acceptancerateisalways1butthismeansslowexploration

• Burn-inisanart!


Exact and Approximate Sampling by Systematic Stochastic Search · approximate, simulation-based inference seems ideal. Exact and approximate sampling are also of intrin-sic interest

Approximate Inference for Fully Bayesian Gaussian Process

PGM 2002/3 – Tirgul6 Approximate Inference: Sampling

Approximate Inference and Side-chain Prediction

APPROXIMATE BAYESIAN PARAMETER INFERENCE FOR DYNAMICAL ...kt.ijs.si/SasoDzeroski/pdfs/2010/2010-Tanevskietal-MANU... · APPROXIMATE BAYESIAN PARAMETER INFERENCE FOR DYNAMICAL SYSTEMS

Approximate MRF Inference Using Bounded Treewidth Subgraphs

1 Approximate Inference 2: Importance Sampling. (Unnormalized) Importance Sampling

Approximate inference for vector parameters

Approximate Inference: MCMC - CSci 5512: Artificial Intelligence IIvision.psych.umn.edu/users/schrater/schrater_lab/courses/... · 2011. 2. 14. · Rejection Sampling Target density

13 Approximate Inference for Observation-Driven Time ...nkantas/aj_nk_ee_obs_time_series_tomacs.pdf · 13 Approximate Inference for Observation-Driven Time Series Models with Intractable

Approximate Inference

Approximate Inference in GMs Samplinglsong/teaching/8803ML/lecture7.pdf · 2012. 2. 2. · Approximate 𝑋∝ Ψ ... Rejection Sampling Create samples like direct sampling ... However,

Approximate genealogical inference

Monte Carlo MCMC: Efficient Inference by Approximate Sampling · 2012. 6. 22. · 1105. 2.2.2 MAP Inference Since MCMC can efciently explore the high density regions for a given distribution,

Inference: Neyman’s Repeated Sampling

Sampling-Based Inference

Approximate Inference in Bayes Nets › courses › csep573 › 11wi › ...Approximate Inference in Bayes Nets Sampling based methods Mausam (Based on slides by Jack Breese and Daphne

Variational Algorithms for Approximate Bayesian Inference

Rollout sampling approximate policy iteration

Global Approximate Inference

Global Approximate Inference Eran Segal Weizmann Institute

07 approximate inference in bn

Sampling Probability and Inference

Bayesian Networks: Sampling Algorithms for Approximate Inference

Approximate Inference in Graphical Models - SLIsli.ics.uci.edu/pmwiki/uploads/Group/forouzan_phd.pdf · Approximate Inference in Graphical Models DISSERTATION submitted in partial

Learning Approximate Inference Policies for Fast Prediction

Approximate inference for the loss-calibrated Bayesianproceedings.mlr.press/v15/lacoste_julien11a/lacoste_julien11a.pdf · Approximate inference for the loss-calibrated Bayesian Simon

Approximate Inference: Samplingcs188/fa18/assets/slides/...Gibbs Sampling Gibbs Sampling Procedure: keep track of a full instantiation x 1, x 2, …, x n. Start with an arbitrary instantiation

Approximate inference: Sampling methods

Approximate inference with Graph Neural Networks...Hence, in practice, they are approximated by techniques like belief propagation, MCMC sampling, and variational inference (Koller

Sampling in Statistical Inference

Parameter Orthogonality and Approximate Conditional Inference

On sampling and approximate counting

Sampling for Inference in Probabilistic Models with …papers.nips.cc/paper/5483-sampling-for-inference-in...Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature

Lecture: Sampling Distributions and Statistical Inferencegwallace/PA_818/Intro to Statistical Inference... · Lecture: Sampling Distributions and Statistical Inference Sampling Distributions