7 workshop on bayesian nonparametricsbnpworkshop.carloalberto.org/files/bnp_abstracts_final.pdf ·...
TRANSCRIPT
7th WORKSHOP ON BAYESIAN NONPARAMETRICS
Moncalieri, June 21–25, 2009
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Contents
Workshop schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 3
Abstracts: Invited Talks
Sunday, June 21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 8
Monday, June 22 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 9
Tuesday, June 23 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 12
Wednesday, June 24 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 16
Thursday, June 25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 20
Abstracts: Poster Presentations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 24
List of participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 46
2
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Workshop schedule
Sunday, June 21
Registration: 12h00–14h25
Opening: 14h25–14h30
Tutorial 1:
Chair: Eugenio Regazzini (University of Pavia)
14h30–16h00 Persi DIACONIS (Stanford University)Partial exchangeability and priors for Markov chains
16h00–16h30 Coffee Break
Tutorial 2:
Chair: Eugenio Regazzini (University of Pavia)
16h30–18h00 Nils HJORT (University of Oslo)Bayesian nonparametrics for survival and event history data
18h00–19h30 Welcome Cocktail
Monday, June 22
Tutorial 3:
Chair: Aad van der Vaart (VU University Amsterdam)
09h15–10h45 Alan GELFAND (Duke University)Bayesian spatial and functional data analysis using gaussian processes
10h45–11h15 Coffee Break
Tutorial 4:
Chair: Aad van der Vaart (VU University Amsterdam)
11h15–12h45 Michael I. JORDAN (University of California Berkeley)Hierarchical nonparametric Bayes with applications
12h45–14h30 Lunch Break
Session 1: 14h30–16h00
Chair: Zoubin Ghahramani (University of Cambridge)
14h30–15h15 Gareth ROBERTS (University of Warwick)Bayesian non-parametric analysis of diffusions
3
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
15h15–16h00 Arnaud DOUCET (The Institute of Statistical Mathematics Tokyo)Particle Markov chain Monte Carlo methods for nonparametric Bayesian time series models
16h00–16h30 Coffee break
Session 2: 16h30–18h00
Chair: Dipak Dey (University of Connecticut)
16h30–17h00 Yongdai KIM (Seoul National University)Bayesian analysis for multi-state event history data
17h00–17h30 Chris C. HOLMES (University of Oxford)Bayesian nonparametric two-sample testing
17h30–18h00 Peter MULLER (University of Texas)A Bayesian semiparametric method for jointly modeling a primary endpoint and longitudinal mea-surements
Tuesday, June 23
Session 3: 09h15–10h45
Chair: Arnaud Doucet (The Institute of Statistical Mathematics Tokyo)
09h15–09h45 Steven N. MacEACHERN (Ohio State University)Nonparametric Bayesian modelling and soft constraints
09h45–10h15 Omiros PAPASPILIOPOULOS (Pompeu Fabra University)Posterior simulation for nonparametric hidden Markov models
10h15–10h45 Subhashis GHOSAL (North Carolina State University)Bayesian multi-scale smoothing of astronomical images using the Chinese restaurant process
10h45–11h15 Coffee Break
Session 4: 11h15–12h45
Chair: Giovanni Peccati (University of Paris X)
11h15–12h00 Aad VAN DER VAART (VU University Amsterdam)On gaussian process priors
12h00–12h45 Judith ROUSSEAU (University of Paris Dauphine)Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametricestimation of the density
12h45–14h30 Lunch Break
Session 5: 14h30–16h00
Chair: Chiara Sabatti (University of California Los Angeles)
14h30–15h15 Peter GREEN (University of Bristol) and Natalia BOCHKINA (University of Edinburgh)Consistency of Bayesian estimators in SPECT and other inverse problems
15h15–16h00 Eugenio REGAZZINI (University of Pavia)Bayesian consistency and classical form of the problem of inverse probabilities
4
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
16h00–16h30 Coffee Break
Session 6: 16h30–17h30
Chair: Gareth Roberts (University of Warwick)
16h30–17h00 Mark STEEL (University of Warwick)Time-dependent stick-breaking processes
17h00–17h30 Yee W. TEH (University College London)The Mondrian process
Poster Session: 17h30–20h00
Poster presenters, titles and abstracts are listed on Pages 24–45
Refreshments will be served during the poster session
Wednesday, June 24
Session 7: 09h15–10h45
Chair: Yongdai Kim (Seoul National University)
09h15–09h45 Fernando QUINTANA (Pontifical Catholic University of Chile)Flexible univariate continuous distributions
09h45–10h15 Lancelot F. JAMES (Hong Kong University of Science and Technology)Lamperti Type Laws
10h15–10h45 Erik SUDDERTH (Brown University)Shared segmentation of natural scenes using dependent Pitman-Yor processes
10h45–11h15 Coffee Break
Session 8: 11h15–12h45
Chair: Albert Y. Lo (Hong Kong University of Science and Technology)
11h15–12h00 Zoubin GHAHRAMANI (University of Cambridge)The Indian buffet process and extensions
12h00–12h45 Francois CARON (INRIA Bordeaux)Random partitions on decomposable graphs
12h45–14h30 Lunch Break
Session 9: 14h30–16h00
Chair: Judith Rousseau (University of Paris Dauphine)
14h30–15h15 Jaeyong LEE (Seoul National University)Sparse Bayesian regression with growing number of covariates
15h15–16h00 Surya TOKDAR (Duke University)Joint linear quantile regression: a semi-parametric Bayesian approach
16h00–16h30 Coffee Break
5
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Session 10: 16h30–18h00
Chair: Maria De Iorio (Imperial College London)
16h30–17h00 Timothy HANSON (University of Minnesota)Some classes of dependent tailfree processes and applications
17h00–17h30 Luis E. NIETO-BARAJAS (ITAM Mexico)Rubbery Polya tree
17h30–18h00 Pietro MULIERE (Bocconi University)Superposition of beta processes
20h30 Social Dinner
Thursday, June 25
Session 11: 09h15–10h45
Chair: Jaeyong Lee (Seoul National University)
09h15–09h45 Jim GRIFFIN (University of Kent)Slice sampling nonparametric models
09h45–10h15 David DUNSON (Duke University)Nonparametric Bayes local mixture models
10h15–10h45 Maria DE IORIO (Imperial College London)Bayesian semiparametric meta-analysis for genetic association studies
10h45–11h15 Coffee Break
Session 12: 11h15–12h45
Chair: Peter Green (University of Bristol)
11h15–12h00 Albert Y. LO (Hong Kong University of Science and Technology)Bayesian subset selection in regression models
12h00–12h45 Michael D. ESCOBAR (University of Toronto)Big alpha
12h45–14h30 Lunch Break
Session 13: 14h30–16h00
Chair: Francois Caron (INRIA Bordeaux)
14h30–15h15 Dipak DEY (University of Connecticut)Semiparametric Bayesian estimation of random coefficients discrete choice models
15h15–16h00 Alessandra GUGLIELMI (Polytechnic Milan)Nonparametric Bayesian mixture modeling for failure time data
16h00–16h30 Coffee Break
Session 14: 16h30–18h00
Chair: Surya Tokdar (Duke University)
6
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
16h30–17h00 Peter HOFF (University of Washington)Generalized marginal likelihoods for semiparametric Bayesian inference
17h00–17h30 Sonia PETRONE (Bocconi University)Bayesian nonparametric regression with temporal constraints
17h30–18h00 Pierpaolo DE BLASI (University of Turin)Bayesian nonparametric estimation and consistency of mixed multinomial logit choice models
7
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Sunday, June 21
14h30–16h00 : Partial exchangeability and priors for Markov chains
Persi DIACONIS (Stanford University)
Abstract: de Finetti’s notions around exchangeability have been generalized in many directions in order to connect Bayesian
theory with statistical practice. I will review the state of the art for Bayesian analysis of Markov chains. This includes
Freedman’s work on partial exchangeability, the natural ”conjugate” on reversible Markov chains coming from random walk
with reinforcement (work with Silke Rolles). It also includes a health set of applications in protein folding.
16h00–16h30 : Coffee Break
16h30–18h00 : Bayesian nonparametrics for survival and event history data
Nils HJORT (University of Oslo)
Abstract: This tutorial will first briefly review some of the more popular processes used for Bayesian nonparametrics, such as
the Dirichlet process, the Beta process, Polya trees, and mixture models, intending a blend of mathematical properties and
statistical motivation. I then go on to methods and models more specifically geared towards applications in the analysis of
survival and event history data. A key ingredient is that of placing priors on the space of hazard rates, often in combination
of covariate information. Illustrations will be given.
18h00–19h30 : Welcome Cocktail
8
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Monday, June 22
09h15–10h45 : Bayesian spatial and functional data analysis using gaussian processes
Alan GELFAND (Duke University)
Abstract: In functional data analysis and, particularly, in spatial data analysis, curves or surfaces are observed, up to
measurement error, at a finite set of locations, for, say, a sample of n individuals. In this setting, a variety of questions
arise. How shall we model these curves? Can we cluster these curves? Can we infer about how different curves are from
each other? Do we seek global clustering or, perhaps, local clustering? Within the world of functional data analysis,
“nonparametric” modeling of such curves typically follows one of two paths. One possibility is that the curves are described
as functions using orthonormal basis representations, typically splines. If these curves are viewed as random then the
coefficients in the representations are random. An alternative nonparametric specification for random curves views them
as realizations of stochastic processes. A convenient stochastic process for modeling such realizations is the Gaussian
process. And, Gaussian processes have been widely used in spatial and functional data analysis. Hence, if we turn
to Bayesian nonparametrics to address the questions above, it seems that we might try to bridge customary Dirichlet
process modeling with Gaussian processes, in particular, using Gaussian process realizations in two ways, first as atoms
but also to enable locally varying weights within the Dirichlet specification. This talk will consider these strategies both
individually and jointly and review modeling in this context, discuss the properties of such models, comment upon compu-
tation to fit such models, and show how they can be used to address the questions above. Illustrative examples will be provided.
10h45–11h15 : Coffee Break
11h15–12h45 : Hierarchical nonparametric Bayes with applications
Michael JORDAN (University of California Berkeley)
Abstract: Hierarchical modeling is a fundamental concept in Bayesian statistics. The basic idea is that parameters
are endowed with distributions which may themselves introduce new parameters, and this construction recurses. Such
constructions are often motivated by considerations of exchangeability or partial exchangeability. In this talk I will discuss the
role of hierarchical modeling in Bayesian nonparametrics, focusing on models in which the infinite-dimensional parameters
are treated hierarchically. For example, I consider a stochastic process in which the base measure G0 for a Dirichlet process
is itself treated as a draw from a Dirichlet process. This yields a natural recursion known as a hierarchical Dirichlet process.
I also discuss hierarchies based on stick-breaking processes and on completely random processes. I demonstrate the value of
these hierarchical constructions in a variety of practical applications, in areas such as computational biology, computer vision
and natural language processing.
12h45–14h30 : Lunch Break
9
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
14h30–15h15 : Bayesian non-parametric analysis of diffusions
Gareth ROBERTS (University of Warwick)
Abstract: This presentation will describe recent progress on Bayesian non-parametric analysis of diffusion drift functions
given continuous data on a finite time interval. It turns out that Gaussian processes can be used as conjugate priors, and we
describe methodology for characterising posterior mean and covariance structure in terms of solutions to differential equations
with coefficients given as functions of the observed diffusion local time.
Co-authors: Yvo Pokern (University of Warwick, UK), Omiros Papaspiliopoulos (Pompeu Fabra University, Spain),
Andrew Stuart (University of Warwick, UK)
15h15–16h00 : Particle Markov chain Monte Carlo methods for nonparametric Bayesian time seriesmodels
Arnaud DOUCET (The Institute of Statistical Mathematics Tokyo)
Abstract: Nonparametric Bayesian models like infinite hidden Markov models or time-varying Dirichlet processes are attractive
time series models. However it remains difficult to perform inference for such models as it requires the use of sophisticated
Markov chain Monte Carlo (MCMC) schemes. We propose a new class of MCMC algorithms relying on particle filtering
proposals. One of the major advantages of this approach is that it offers a systematic methodology in order to construct
efficient high-dimensional proposal distributions whilst requiring the practitioner to design only low-dimensional proposal
distributions. It offers the possibility to simultaneously update very large vectors of dependent random variables. We
demonstrate the performance of this methodology on various examples.
16h00–16h30 : Coffee Break
16h30–17h00 : Bayesian analysis for multi-state event history data
Yongdai KIM (Seoul National University)
Abstract: Multi-state event history data are frequently met in survival analysis where there are several types of events a
subject can experience. Examples are competing risks model and illness- death model, to name just a few. And, Markov
processes are popularly used to model mulit-state event history data. In this talk, I propose a new prior process for the
cumulative intensity function (CIF) of a Markov process. Hjort (1990) and Kim (1999) used independent beta processes for a
prior of the CIFs. However, independent beta processes may not be a valid prior for the CIFs when two or more CIFs have
jumps at the same time. We resolve this problem to introduce a beta-Dirichlet process which is a multivariate subordinator
(nondecreasing process with independent increments). We prove that beta-Dirichlet processes are conjugate. Also, I explain
how to apply beta-Dirichlet processes prior to a Bayesian semi-parametric regression model. For illustration, I analyze a credit
history data.
10
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
17h00–17h30 : A Bayesian semiparametric method for jointly modeling a primary endpoint and longitu-dinal measurements
Peter MULLER (University of Texas)
Abstract: We consider inference for data from a clinical trial of treatments for metastatic prostate cancer. Patients joined
the trial with diverse prior treatment histories. The resulting heterogeneous patient population gives rise to challenging
statistical inference problems when trying to predict time to progression on different treatment arms. Inference is further
complicated by the need to include a longitudinal marker as a covariate. To address these challenges, we develop a
semi-parametric model for joint inference of longitudinal data and an event time. The proposed approach includes the
possibility of cure for some patients. The event time distribution is based on a non-parametric Polya tree prior. For the
longitudinal data we assume a mixed effects model. Incorporating a regression on covariates in a non-parametric event
time model in general, and for a Polya tree model in particular, is a challenging problem. We exploit the fact that the
covariate itself is a random variable. We achieve an implementation of the desired regression by factoring the joint model
for the event time and the longitudinal outcome into a marginal model for the event time and a regression of the longitudi-
nal outcomes on the event time, i.e., we implicitly model the desired regression by modeling the reverse conditional distribution.
Co-authors: Song Zhang, Kim-Ahn Do
17h30–18h00 : Bayesian nonparametric two-sample testing
Chris HOLMES (University of Oxford)
Abstract: We consider Bayes and Empirical Bayes two-sample nonparametric tests using Polya Tree priors. We show that the
Polya tree is an attractive nonparametric model for this setting as it allows for exact analytic expression of the Bayes Factor,
of the evidence of two generating processes (distributions) rather than one, without the need to artificially truncate the level
of the tree. Moreover Bayes Factors are easily computable either in closed form or via simple one-dimensional quadratures.
We compare the sensitivity of the Bayes tests against traditional non-Bayesian approaches, such as Kolmogorov-Smirnov, on
a number of examples.
11
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Tuesday, June 23
09h15–09h45 : Nonparametric Bayesian modelling and soft constraints
Steven MacEACHERN (Ohio State University)
Abstract: The versatility of nonparametric Bayesian techniques is illustrated by their use in two strongly conflicting fashions:
for exploratory data analysis and for modelling a data-generating process. Exploratory analyses seek to avoid imposing
structure on the data, while traditional modelling focuses on the imposition of structure. Far from being a bad thing,
imposition of structure allows us to express our knowledge about features (often latent) of probability models. Nonparametric
methods allow us to replace the stringent constraints that are generally imposed with soft constraints. In this talk, I will
describe a selection of core modelling concepts and how soft-constraint versions of them can be captured with nonparametric
Bayesian methods.
Co-authors: Juhee Lee, Zhen Wang, Catherine Forbes
09h45–10h15 : Posterior simulation for nonparametric hidden Markov models
Omiros PAPASPILIOPOULOS (Pompeu Fabra University)
Abstract: Dirichlet process mixtures (MDPs), are now standard in semiparametric modelling. Posterior inference for such
models is typically performed using Markov chain Monte Carlo methods, which can be roughly categorised into marginal and
conditional methods. The former integrate out analytically the infinite-dimensional component of the hierarchical model and
sample from the marginal distribution of the remaining variables using the Gibbs sampler. Conditional methods impute the
Dirichlet process and update it as a component of the Gibbs sampler. Since this requires imputation of an infinite-dimensional
process, implementation of the conditional method has relied on finite approximations.
In the first part of the talk we show how to avoid such approximations by novel Gibbs sampling algorithms which sample from
the exact posterior distribution of quantities of interest. The approximations are avoided by the technique of retrospective
sampling. Motivated by the modelling of copy number variation (CNVs) in the human genome we have developed hidden
Markov models where the likelihood is given by an MDP. We term the resulting model an HMM-MDP model. Thus, we deal
with a model with two levels of clustering for the observed data, a temporally persisting (local) clustering induced by the
HMM and a global clustering induced by the Dirichlet process. The second part of the talk shows how to design efficient
conditional methods for fitting these models elaborating on the methods developed on the first part of the talk and on
dynamic programming techniques.
12
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
10h15–10h45 : Bayesian multi-scale smoothing of astronomical images using the Chinese restaurantprocess
Subhashis GHOSAL (North Carolina State University)
Abstract: Astronomical images often consist of pixel-by-pixel photon count data, especially the X-ray images of distant
stars and supernovas taken recently by the Chandra telescope. The photon counts can be adequately modeled by the
Poisson distribution. In order to estimate the underlying intensity parameters, we consider a Bayesian multi-scale approach
to smoothing. In the frequentist literature, wavelet-like smoothing methods such as Haar wavelet transform, wedgelets and
platelets, as well as penalized likelihood methods, have been considered. In the Bayesian literature, multi-scale methods
based on parent-child group of four with Dirichlet distribution controlling prior mass allocation, have been investigated. In
this presentation, we consider a similar approach, but we allow formation of ties through a Chinese restaurant process, and
then let prior mass be allocated through an appropriate Dirichlet distribution conditional on the obtained configurations. By
allowing ties, we are able detect structures more often in images. All parameters in our procedure are data driven. We also
draw independent samples from the posterior distribution directly, thanks to the near-analytic expressions and sequential
factorizations of posterior probabilities into various parent-child group of blocks of pixels. As a result, we avoid Markov Chain
Monte-Carlo draws, saving computing time by several orders of magnitude. We also show that the resulting method has
important consistency properties as the exposure (total flux) goes to infinity: the posterior distribution concentrates in the
neighborhood of true intensity, superfluous structures vanish asymptotically while genuine structures show up in the process.
In order to compare our methodology to the current methods, a simulation example using a photon-limited image source was
conducted. Photon-limited images are common in astronomical imaging due to faint sources or limited time of exposure. We
observe that our method outperforms the older Bayesian methods and wavelet-based frequentist methods, while performing
nearly like the platelet method. On the other hand, the platelet method takes an enormous time to compute, especially if the
total photon count is considerably large, while the computing time of our method does not increase with the total flux. We
apply our method on some of the recent photos taken by the Chandra X-ray observatory.
Co-authors: John T. White (North Carolina State University, USA)
10h45–11h15 : Coffee Break
11h45–12h00 : On gaussian process priors
Aad VAN DER VAART (VU University Amsterdam)
Abstract: A sample path of a Gaussian process can serve as a prior for an unknown function, for instance a regression
function, or, after exponentiation and renormalization, a density function. Because any covariance function defines a Gaussian
process, such priors offer great flexibility. In this talk we discuss how the covariance function determines the rate of contraction
of the posterior distribution, in the usual frequentist set-up where the observations are sampled from a fixed distribution.
The reproducing kernel Hilbert space attached to the covariance function plays the crucial role. One particular example we
discuss is an infinitely smooth Gaussian process, with time index rescaled by an independent Gamma variable. This prior can
be shown to adapt to the unknown smoothness of a function.
Co-author: Harry van Zanten (VU University Amsterdam, Netherlands)
13
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
12h00–12h45 : Rates of convergence for the posterior distributions of mixtures of betas and adaptivenonparamatric estimation of the density
Judith ROUSSEAU (University of Paris Dauphine)
Abstract: In this work we investigate the asymptotic properties of nonparametric bayesian mixtures of Betas for estimating
a smooth density on [0,1]. We consider a parameterisation of Betas distributions in terms of mean and scale parameters
and construct a mixture of these Betas in the mean parameter, while putting a prior on this scaling parameter. We prove
that such Bayesian nonparametric models have good frequentist asymptotic properties. We determine the posterior rate
of concentration around the true density and prove that it is the minimax rate of concentration when the true density
belongs to a Holder class with regularity β, for all positive β, leading to a minimax adaptive estimating procedure of
the density. We show that Bayesian kernel estimation is more flexible than the usual frequentist kernel estimation allow-
ing for adaptive rates of convergence, using a simple trick which can be used in many other types of kernel Bayesian approaches.
14h30–15h15 : Lunch Break
14h30–15h15 : Consistency of Bayesian estimators in SPECT and other inverse problems
Peter GREEN (University of Bristol) and Natalia BOCHKINA (University of Edinburgh)
Abstract: Formulating a statistical inverse problem as one of inference in a Bayesian model has great appeal, notably for
what this brings in terms of coherence, the interpretability of regularisation penalties, the integration of all uncertainties,
and the principled way in which the set-up can be elaborated to encompass broader features of the context, such as
measurement error, indirect observation, etc. The Bayesian formulation comes close to the way that most scientists
intuitively regard the inferential task, and in principle allows the free use of subject knowledge in probabilistic model
building. However, in some problems where the solution is not unique, for example in ill-posed inverse problems, it is
important to understand the relationship between the chosen Bayesian model and the resulting solution. Taking emission
tomography as a canonical example for study, we will present results about consistency of the posterior distribution of
the reconstruction, and discuss how our method may be used to shed light on a broader class of posterior convergence problems.
15h15–16h00 : Bayesian consistency and classical form of the problem of inverse probabilities
Eugenio REGAZZINI (University of Pavia)
Abstract: The classical problem of the inverse probabilities can be viewed as an ancestor of the more fashionable problem of
the consistency of posterior distributions. The relationship between them has already been pointed out and explained in a
paper of Diaconis and Freedman [The Annals of Statistics, 18, 1317-1327 (1990)]. With this talk we would like to highlight,
on the one hand, the differences between the two problems and to give, on the other hand, an outline of possible theoretical
developments of the former.
Co-authors: Emanuele Dolera (University of Pavia, Italy)
16h00–16h30 : Coffee Break
14
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
16h30–17h00 : Time-dependent stick-breaking processes
Mark STEEL (University of Warwick)
Abstract: This paper considers the problem of defining a time-dependent nonparametric prior. A recursive construction
allows the definition of priors whose marginals have a general stick-breaking form. The processes with Poisson-Dirichlet
and Dirichlet process marginals have interesting interpretations that are further investigated. We develop a general
conditional Markov Chain Monte Carlo (MCMC) method for inference in the wide subclass of these models where the
parameters of the stick-breaking process form increasing sequences. We derive a Polya urn scheme type representation of
the Dirichlet process construction, which allows us to develop a marginal MCMC method for this case. The results section
shows the relative performance of the two MCMC schemes for the Dirichlet process case and contains three real data examples.
Co-authors: Jim Griffin (University of Kent, UK)
17h00–17h30 : The Mondrian process
Yee W. TEH (University College London)
Abstract: We describe a novel class of distributions, called Mondrian processes, which can be interpreted as probability
distributions over kd-tree data structures. Mondrian processes are multidimensional generalizations of Poisson processes and
this connection allows us to construct multidimensional generalizations of the stick-breaking process described by Sethuraman
[1994], recovering the Dirichlet process in one dimension. After introducing the Aldous-Hoover representation for jointly and
separately exchangeable arrays, we show how the process can be used as a nonparametric prior distribution in Bayesian models
of relational data.
17h30–20h00 : Poster Session
15
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Wednesday, June 24
09h15–09h45 : Flexible univariate continuous distributions
Fernando QUINTANA (Pontifical Catholic University of Chile)
Abstract: Based on a constructive representation, which distinguishes between a skewing mechanism P and an underlying
symmetric distribution F , we introduce two flexible classes of distributions. They are generated by nonparametric modelling
of either P or F . We examine properties of these distributions and consider how they can help us to identify which aspects
of the data are badly captured by simple symmetric distributions. Within a Bayesian framework, we investigate useful prior
settings and conduct inference through MCMC methods. On the basis of simulated and real data examples, we make
recommendations for the use of our models in practice. Our models perform well in the context of density estimation for data
exhibiting multimodal features, and also for regression modelling.
Co-authors: J.T. Ferreira (Endeavour Capital Management, UK), Mark Steel (University of Warwick, UK)
09h45–10h15 : Lamperti type laws
Lancelot JAMES (Hong Kong University of Science and Technology)
Abstract: I introduce a class of random variables I call Lamperti Type Laws. These variables turn out to be special cases of
Dirichlet means and are intimately connected with two parameter Poisson Dirichlet process otherwise known as Pitman-Yor
processes. We show how these are used to achieve explicit results for the time spent positive of generalized bessel bridges on
certain random subsets of [0,1].
10h15–10h45 : Shared segmentation of natural scenes using dependent Pitman-Yor processes
Erik SUDDERTH (Brown University)
Abstract: We explore statistical frameworks for the simultaneous, unsupervised segmentation and discovery of visual object
categories from image databases. Examining a large set of manually segmented scenes, we show that object frequencies
and segment sizes both follow power law distributions, which are well modeled by the Pitman-Yor (PY) process. This
generalization of the Dirichlet process leads to learning algorithms which discover an unknown set of objects, and segmentation
methods which automatically adapt their resolution to each image. Generalizing previous applications of PY priors, we use
non-Markov Gaussian processes to infer spatially contiguous segments which respect image boundaries. Using a novel family
of variational approximations, our approach produces segmentations which compare favorably to state-of-the-art methods,
while simultaneously discovering categories shared among natural scenes.
10h45–11h15 : Coffee Break
16
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
11h15–12h00 : The Indian buffet process and extensions
Zoubin GHAHRAMANI (University of Cambridge)
Abstract: Much work in nonparametric Bayesian statistics focuses on the Dirichlet process (DP) and its associated combi-
natorial object, the Chinese restaurant process (CRP). The DP and CRP have found important uses in mixture modelling,
allowing inference in models with a countable but unbounded number of mixture components. In analogy to CRPs, we have
recently developed the Indian buffet process (IBP) which defines probability distributions on binary matrices with exchangeable
rows and an unbounded number of columns. The IBP makes it possible to define and do inference in models with an
unbounded number of latent variables. I will review properties of the IBP, inference algorithms, and a number of applications,
including: sparse latent factor and independent components models, time series models with an unbounded number of
hidden processes, and nonparametric matrix factorisation models. Time permitting, I will describe recent extensions of the IBP.
12h00-12h45 : Random partitions on decomposable graphs
Francois CARON (University of British Columbia)
Abstract: Probabilistic data clustering has numerous applications in machine learning and statistics. Formally, we associate
to each data a latent allocation variable. These latent variables can share the same value and induce a partition of the
data. In a Bayesian setting, the partition is assumed random and we set a prior distribution on it. Models with both a fixed
or unknown number of clusters have been considered in the literature. In particular, Dirichlet multinomial allocation and
Dirichlet process partition models have become very popular over the past few years. We propose here extensions of these
models to decomposable graphical models. These models have appealing properties and can be fitted using Markov chain
Monte Carlo and Sequential Monte Carlo algorithms.
12h45–14h30 : Lunch Break
14h30–15h15 : Sparse Bayesian regression with growing number of covariates
Jaeyong LEE (Seoul National University)
Abstract: In this talk, we consider the Bayesian regression model when the number of covariates is increasing as the number
of observations tends to infinity. We consider the posterior and Bayes estimators along with the g-type prior and variable
selection prior. Unless the number of covariates grows slowly, the posterior and Bayes estimators with the g-type prior are all
inconsistent. The posterior and Bayes estimators with variable selection prior are shown to be all consistent. A simulation
study also confirm the theoretical results.
17
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
15h15–16h00 : Joint linear quantile regression: a semi-parametric Bayesian approach
Surya TOKDAR (Duke University)
Abstract: Quantile regression is a natural tool to analyze trends of the extreme realizations of a recurring event. While a
number of linear quantile regression techniques exist in the literature, none provides a satisfactory calibration of the trend.
These techniques employ a separate analysis for each quantile – and it is often hard to reconcile the ensuing inferences. A joint
inference requires modeling the overall conditional behavior of the variable of interest. Unfortunately, methods which deduce
quantile curves by estimating the conditional density semi-parametrically, cannot perform trend analysis in a meaningful way
due to the non-linearity of the estimated quantile curves. Moreover, the fit of such conditional density models depends mostly
on the central observations making inference on tails somewhat unreliable.
We propose a joint model for all conditional quantile curves of a variable of interest given covariate values. Each curve is
constrained to be linear. Any such model can be induced through two function-valued parameters each of which provides a
non-decreasing map of the unit interval onto the range of the variable of interest. We use a Gaussian process prior to jointly
model these two functions. Model fitting is done through an efficient MCMC sampler. Numerical simulations are provided to
illustrate performance of this method. We also demonstrate its application to analyzing intensity trends of tropical cyclones –
a problem that motivated this study.
16h00–16h30 : Coffee Break
16h30–17h00 : Some classes of dependent tailfree processes and applications
Tim HANSON (University of Minnesota)
Abstract: Priors over related distributions have received increased attention in the Bayesian literature. Recent papers include
De Iorio, Mueller, Rosner, and MacEachern (2004), Gelfand, Kottas, and MacEachern (2005), Griffin and Steel (2006),
Dunson, Pillai, and Park (2007), Reich and Fuentes (2007), and many others. All of these approaches generalize some aspect
of a stick-breaking prior, either by including weights that change with covariate levels (which could be time or space), atoms
that change with covariate levels, or both. These processes are typically convolved with a continuous, usually normal kernel
yielding a smooth density at each covariate level.
In this talk I will discuss some alternative nonparametric dependent processes: tailfree priors where conditional probabilities
are stochastic processes or simple parametric functions indexed by covariates. Fixing the partition across covariate levels
simplifies computations and allows analyses of rather large, censored data sets. Data applications include a generalization of
the accelerated failure time model where parameters retain interpretability in terms of median survival, an example of growth
curve analyses, and a Rasch model applied to educational testing data where the student-specific random effects distribution
changes with levels of covariates.
17h00-17h30 : Rubbery Polya tree
Luis E. NIETO-BARAJAS (ITAM Mexico)
Abstract: Polya trees (PT) are random probability measures (RPM) which can assign probability one to the set of continuous
distribution for certain specifications of the hyper-parameters. This feature distinguishes the PT from the popular Dirichlet
process (DP) model which assigns probability one to the set of discrete distributions. However, the PT is not nearly as
widely used as the DP prior. Probably the main reason is an awkward dependence of posterior inference on the choice of
the partitioning subsets in the definition of the PT. In particular this dependence implies discontinuities of the density of the
estimated RPM at the boundaries of the partitioning subsets. We propose a generalization of the PT prior that mitigates this
undesirable dependence on the partition structure. The proposed new process is not a PT anymore, but still is a tail free process.
Co-authors: Peter Mueller (University of Texas, USA)
18
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
17h30–18h00 : Superposition of beta processes
Pietro MULIERE (Bocconi University)
Abstract: We define and investigate a new neutral to the right random distribution function on the positive real line termed
generalized beta-Stacy process. It corresponds to the superposition of independent beta processes at the cumulative hazard
level. The definition is constructive and starts with a discrete time version in terms of random probability masses obtained
from suitably defined products of independent beta distributed random variables. The continuous time version is derived as
the corresponding infinitesimal weak limit. We provide interpretation in terms of placing a prior on the survival distribution
arising from independent competing failure times and we illustrate posterior inference on a real data example.
Co-authors: Pierpaolo De Blasi (University of Turin, Italy), Stefano Favaro (University of Turin, Italy)
20h30 : Social Dinner
19
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Thursday, June 25
09h15–09h45 : Slice sampling nonparametric models
Jim GRIFFIN (University of Kent)
Abstract: Slice sampling methods introduce latent variables to make MCMC computation easier. In nonparametric models,
these latent variables are designed to make the model conditionally finite. This talk discusses the application of this method
to mixture models with a range of priors for the mixing measure including stick-breaking and normalized random measures.
The performance on different models is compared with other competing methods.
09h45–10h15 : Nonparametric Bayes local mixture models
David DUNSON (Duke University)
Abstract: Mixture models have become extremely widely used for a broad range of problems, including density estimation,
clustering and classification. Most discrete mixture models make the implicit assumption that an individual is allocated
to the same cluster id for all of their parameters. This leads to global clustering of subjects, with two subjects that are
assigned to the same cluster for one of their parameters automatically assigned to the same cluster for all their parameters.
This talk considers nonparametric Bayes local partition processes, which allow sparse local mixture modeling of unknown
multivariate distributions. We propose a local partition process (LPP) prior based on a carefully-defined mixture of global and
independent allocation processes. The proposed methods rely on the generation of infinitely-many p x 1 unique coefficient
vectors (UCVs). Global mixture models are based on allocating an individual randomly to one of these UCVs, while the LPP
allows individuals to be allocated to different UCVs for each of their parameters. Some properties are described and an exact
block Gibbs sampler is developed for posterior computation avoiding truncation of the infinite random measure. The methods
are illustrated using simulated data, a functional data example, and an application of semiparametric sparse latent factor
models for predicting impending sickness from gene expression measurements.
10h15–10h45 : Bayesian semiparametric meta-analysis for genetic association studies
Maria DE IORIO (Imperial College London)
Abstract: We present a Bayesian nonparametric model for the meta-analysis of candidate gene studies with a binary outcome.
Such studies often report results from association tests for different, possibly study- specific and non- overlapping markers
(typically SNPs) in the same genetic region. Meta-analyses of the results at each marker in isolation are seldom appropriate as
they ignore the correlation that may exist between markers due to linkage disequilibrium (LD) and cannot assess the relative
importance of variants at each marker. Also such marker-wise meta-analyses are restricted to only those studies that have
typed the marker in question, with a potential loss of power. A better strategy is one which incorporates information about the
LD between markers so that any combined estimate of the effect of each variant is corrected for the effect of other variants,
as in multiple regression. Here we develop a Bayesian nonparametric model which models the observed genotype group
frequencies conditional to the case/control status and uses pairwise LD measurements between markers as prior information
to make posterior inference on adjusted effects. The approach allows borrowing strength across studies and across markers.
The analysis is based on a mixture of Dirichlet processes model as the underlying semiparametric model. Full posterior in-
ference is performed through Markov chain Monte Carlo algorithms. The approach is demonstrated on simulated and real data.
10h45–11h15 : Coffee Break
20
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
11h15–12h00 : Bayesian subset selection in regression models
Albert Y. LO (Hong Kong University of Science and Technology)
Abstract: The selection of predictors to include is an important problem in building a multiple regression model. The Bayesian
approach to this problem simply converts the prior distribution on the possible subsets to a posterior distribution and is
desirable. The approach often assumes a normal error, which is a restriction. This paper uses the Bayesian mixture method to
relax this restriction to allow for seemingly more realistic errors that are unimodal and/or symmetric. The main thrust of this
method essentially reduces an infinite-dimensional stochastic process problem of averaging random distributions to a finite-
dimensional one based on averaging random partitions. The posterior distribution of the parameters is an average of random
partitions. Nesting a Metropolis-Hastings algorithm within a weighted Chinese restaurant process of sampling partitions results
in a MCMC, which provides a stochastic approximation to the posterior mode of the parameters. Numerical examples are given.
Co-authors: Baoqian Pao
12h00–12h45 : Big alpha
Michael D. ESCOBAR (University of Toronto)
Abstract: The Dirichlet process has two parameters. One is a distribution and it can be consider a location parameter.
Sampled distributions from the Dirichlet process are around this central distribution. The other parameter, sometimes denoted
alpha, is a scaler and is related to how far away the sampled distributions are to the central distribution. For most of the
past 20 years, most uses of the Dirichlet process and its relatives like the Dirichlet process mixture have concentrated on the
case when alpha is small. For example, when alpha is small, the Dirichlet process mixture is performing a type of Bayesian
finite mixture analysis. However, there are some conditions when one would consider situations when alpha is big. This talk
will first look at some old examples which would motivate this situation and then discuss how this might be very useful if one
wants to consider hierarchical models and ”borrow strength” in larger ”nonparametric” models.
12h45–14h30 : Lunch Break
14h30–15h15 : Semiparametric Bayesian estimation of random coefficients discrete choice models
Dipak K. DEY (University of Connecticut)
Abstract: Heterogeneity in choice models is typically assumed to have a normal distribution in both Bayesian and classical
setups. In this talk, we propose a semiparametric Bayesian framework for the analysis of random coefficients discrete choice
models that can be applied to both individual as well as aggregate data. Heterogeneity is modeled using a Dirichlet process
prior which varies with consumers characteristics through covariates. We develop a Markov chain Monte Carlo algorithm for
fitting such model, and illustrate the methodology using two different datasets: a household level panel dataset of peanut
butter purchases, and supermarket chain level data for 31 ready-to-eat breakfast cereals brands.
Co-authors: Sylvie Tchumtchoua (University of Connecticut, USA)
21
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
15h15–16h00 : Nonparametric Bayesian mixture modeling for failure time data
Alessandra GUGLIELMI (Polytechnic Milan)
Abstract: We examine a nonparametric Bayesian mixture of parametric densities on the positive reals mixed by a (normalized)
generalized gamma process (Brix, 1999). This class of mixtures (NGG mixtures), which can be described in terms of a
couple of positive parameters (sigma,k), encompasses the Dirichlet process mixture (DPM) model, but it is more flexible
in the detection of clusters in the data, as far as density estimation is concerned. MCMC algorithms estimating linear and
nonlinear functionals of the predictive distributions can be built. The best-fitting mixture is found by minimizing a Bayes
factor for the ”mean” parametric distribution against the non-parametric alternative, or computing, under a full Bayesian
model specification, the posterior distribution for (sigma,k).
As an application, we consider an accelerated failure time (AFT) model for univariate failure times of Kevlar fibres from
different spools (with right censoring), where the error is represented as a NGG mixture; the number of components in the
mixture can be interpreted as the number of random effects. The advantage over the Bayesian ”traditional” parametric
random-effects models is that this number can be inferred from the data. Compared to previous results, we obtain narrower
interval estimates of the quantiles and also useful credibility intervals for the predictive survival functions.
16h00–16h30 : Coffee Break
16h30–17h00 : Generalized marginal likelihoods for semiparametric Bayesian inference
Peter HOFF (University of Washington)
Abstract: Estimation in the presence of a high-dimensional nuisance parameter can be difficult, both theoretically and
practically. In some problems, a statistic can be obtained whose distribution depends only on the parameter of interest,
and not on the nuisance parameter. In these situations a likelihood based on the statistic can be constructed and used for
estimation. The use of such a marginal likelihood does not require estimation of or a prior distribution for the nuisance
parameter.
In this talk we present a generalization of marginal likelihood, in which the information used to estimate the parameter of
interest may or may not be in the form of a statistic. We discuss conditions under which the information is maximally infor-
mative, and the general properties of estimators based on such information. Example applications, including semiparametric
copula estimation and multinomial choice models, are discussed.
17h00–17h30 : Bayesian nonparametric regression with temporal constraints
Sonia PETRONE (Bocconi University)
Abstract: Motivated by a problem in finance, I discuss a Bayesian nonparametric approach to dynamic regression, when one
has constraints on the temporal evolution of the regression curve. More specifically, the curve is supposed to be the solution
of a stochastic differential equation. Usually, an analytical solution is not available, so, in a Bayesian approach, it is treated
as random. A nonparametric prior is proposed on the family of random curves over time, taking into account the constraints
implied by the SDE. Estimation is discussed for a discretized version of the model.
22
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
17h30–18h00 : Bayesian nonparametric estimation and consistency of mixed multinomial logit choicemodels
Pierpaolo DE BLASI (University of Turin)
Abstract: This paper develops nonparametric estimation for discrete choice models based on the Mixed Multinomial
Logit (MMNL) model. It has been shown that MMNL models encompass all discrete choice models derived under the
assumption of random utility maximization, subject to the identification of an unknown distribution G. Noting the mixture
model description of the MMNL, we employ a Bayesian nonparametric approach, using nonparametric priors on the
unknown mixing distribution G, to estimate choice probabilities. We provide an important theoretical support for the use
of the proposed methodology by investigating consistency of the posterior distribution for a general nonparametric prior
on the mixing distribution. Consistency is defined according to a L1-type distance on the space of choice probabilities
and is achieved by extending to a regression model framework a recent approach to strong consistency based on the
summability of square roots of prior probabilities. Moving to estimation, slightly different techniques for non-panel and
panel data models are discussed. For practical implementation, we describe efficient and relatively easy to use blocked
Gibbs sampling procedures. These procedures are based on approximations of the random probability measure by classes
of finite stick-breaking processes. A simulation study is also performed to investigate the performance of the proposed methods.
Co-authors: Lancelot F. James (Hong Kong University of Science and Technology, HK), John W. Lau (University of
Western Australia, Australia)
23
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Posters (Tuesday, 17h30–20h00)
Presenters and titles
Isadora ANTONIANO VILLALOBOS (University of Kent)Bayesian inference for diffusions based on exact simulation
Raffaele ARGIENTO (IMATI-CNR Milan)A simulation-based approach to full Bayesian inference for mixture densities under the normalized generalized gammaprior
Eric BARAT (CEA-LIST) & Mame Diarra FALL (L2S/SUPELEC)Pitman-Yor Mixtures prior for nonparametric spatial emission tomography
Anirban BHATTACHARYA (Duke University)Bayesian local mixtures of factor analyzers
Abhishek BHATTACHARYA (Duke University)Nonparametric Bayesian density estimation on manifolds with applications to planar shapes
Eunice CAMPIRAN (UNAM Mexico)How to use product partitions models to reflect the prior knowledge of the stratification in finite populationsampling
Alessandro CARTA (University of Warwick)Modelling multi-output stochastic frontier using copulas
Annalisa CERQUETTI (Collegio Nuovo Pavia)On a class of Bayesian nonparametric priors derived by subordination of stable processes
James M. CIERA (University of Padua)Fast approximate Bayesian functional mixed effects model
Jose C.S. DE MIRANDA (University of Sao Paulo)Proxy maximum probability estimation of Poisson intensities
Chang DOREA (Universidade de Brasilia)On the robustness of Bayesian modelling of location and scale structures using heavy-tailed distributions
Ilenia EPIFANI (Politecnico di Milano)Priors for vectors of survival functions
Marian FARAH (University of California Santa Cruz)Bayesian nonparametric modeling of cross-section vs. LET for the prediction of on-orbit upset rate
Emily FOX (MIT)Sharing features among dynamical systems with beta processes
Kassandra FRONCZYK (University of California Santa Cruz)Nonparametric Bayesian regression for replicated categorical responses
Dimitros GIANNIKIS (Athens University of Economics and Business)Clustering of many financial time series using nonparametric Bayesian technics
Dilan GORUR (University College London)
24
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Efficient sequential Monte Carlo for inference on Kingman’s coalescent
Spyridon HATJISPYROS & Theodoros NICOLERIS (University of the Aegean)Dependent mixtures of Dirichlet processes
Daniel HEINZ (Carnegie Mellon University)Non-parametric hyper Markov priors
Ricardo HENAO (Technical University of Denmark)Robust processes for latent variables in dynamical factor models
Gudmund HERMANSEN (University of Oslo)Bayesian nonparametric modelling of covariance functions, with application to time series and spatial statistics
Zhaowei HUA (University of North Carolina at Chapel Hill)Semiparametric Bayes local additive models for longitudinal data
Alejandro JARA (Universidad de Concepcion)A novel class of mixtures of multivariate Polya trees
Mark JENSEN (Federal Reserve Bank of Atlanta)Bayesian semiparametric stochastic volatility modeling
Maria KALLI (University of Kent)Mixtures of stick-breaking processes
George KARABATSOS (University of Illinois-Chicago)A Bayesian nonparametric causal model
Dohyun KIM (Seoul National University)Bayesian subspace clustering with Indian Buffet process
Gwangsu KIM (Seoul National University)The time varying survival analysis by Bayesian bootstrap
Michalis KOLOSSIATIS (University of Warwick)Bayesian nonparametric modelling of grouped data with an application to stochastic frontiers
Willem KRUIJER (University Paris Dauphine)Adaptive Bayesian density estimation with location-scale mixtures
Heng LIAN (Nanyang Technological University)Posterior convergence and model estimation in Bayesian change-point problems
Claudio MACCI (University of Rome Tor Vergata)Large deviations for Bayesian estimators in first-order autoregressive processes
Juan-Carlos MARTINEZ-OVANDO (University of Kent)An alternative nonparametric time-series model
Takashi MATSUMOTO (Waseda University)Dirichlet process EM algorithm for semi-supervised learning
Andriy NORETS (Princeton University)Bayesian modeling of joint and conditional distributions
Debdeep PATI (Duke University)Bayesian nonparametric regression with an unknown predictor-dependent residual distribution
25
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Maria JOAO POLIDORO (Instituto Politecnico Porto)A Bayesian methodology for models adequacy
Cecila PROSDOCIMI (Univeristy of Padua)Countable mixtures of Markov chains
Sandra RAMOS (ISEP)Semiparametric Bayesian approach to gene profile classification
Eva RICCOMAGNO (University of Genova)Causal analysis with chain event graphs
Abel RODRIGUEZ (University of California Santa Cruz)Nested partition models
Carlos RODRIGUEZ (University of Kent)Sampling the mixture normal model
Daniel ROY (MIT)On the computability of de Finetti measures
Matteo RUGGIERO (University of Pavia)Gibbs sampling the two-parameter Poisson-Dirichlet process and its diffusion limit
Catia SCRICCIOLO (Bocconi University)Rates of convergence for Bayes and ML estimators of mixtures of exponential power densities
Dario SPANO (University of Warwick)Genealogy and dual processes associated with time-dependent Dirichlet processes, and their polynomial eigen-functions
Matthew TADDY (University of Chicago)Conditional modeling for survival data
Mahlet TADESSE (Georgetown University)Bayesian nonparametric model for integrating genomic data sets
Takaaki TOKUDA (Waseda University)Maximum a posteriori estimation for Dirichlet process language models
Sinead WILLIAMSON (University of Cambridge)A sparse, infinite topic model based on the Indian Buffet process
Fei XIANG (University of Kent)Bayesian consistency for regression model
Hao WU (Free University of Berlin)Artificial Conditional Dirichlet Processes
Oliver ZOBAY (University of Bristol)Mean field inference for the Dirichlet process mixture model
26
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Abstracts
Bayesian inference for diffusions based on exact simulation
Isadora ANTONIANO VILLALOBOS (University of Kent)
Abstract: When a certain phenomena is modelled by means of a real-valued diffusion process, the model is often stated in
terms of a stochastic differential equation. Statistical inference is then aimed at the estimation of the parameters appearing
in the drift and diffusion coefficients of the SDE. Exact simulation has been used for Monte Carlo Maximum Likelihood
estimation. It has the clear advantage of being free of any approximation or discretization error. However, little has been
done in the direction of Bayesian analysis. The only method proposed consists of independent exact simulation of the path
between data points. Therefore, inference is conditioned on the unique specific path observed and the number of simulation
steps involved in each Markov Chain iteration grows with the size of the data set.
We propose an alternative way of using the exact simulation algorithm for Bayesian estimation of the parameters of a specific
family of SDEs. Our method allows for a data set containing points from more than one observed path and is performed on a
fixed time interval, so its computational complexity does not grow with the number of observations.
Co-author: Stephen Walker (University of Kent)
A simulation-based approach to full Bayesian inference for mixture densities under the normalizedgeneralized gamma prior
Raffaele ARGIENTO (CNR-IMATI Italy)
Abstract: We consider a mixture of parametric densities with a normalized generalized gamma (NGG) process (Brix, 1999)
as mixing measure. This process is an almost surely discrete random probability measure encompassing the Dirichlet one.
The NGG process is identified by a parametric distribution and a pair of positive parameters (sigma,kappa). In order to
improve the posterior estimate of the number of components of the mixture we assume a bivariate prior distribution for
(sigma,kappa). Similarly to Nieto-Barajas and Pruenster (2008), we built a Gibbs sampler algorithm involving an almost sure
approximation of the posterior distribution of the mixing process, investigating also the convergence of the approximated
functionals to the true ones. In this way we will pursue a full nonparametric Bayesian inference, obtaining posterior estimates of
linear and non linear functionals of the population distribution. We illustrate our results through a problem of density estimation.
Co-authors: Alessandra Guglielmi (Polytechnic Milan), Antonio Pievatolo (CNR-IMATI Italy)
27
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Pitman-Yor mixtures prior for nonparametric spatial emission tomography
Eric BARAT (CEA-LIST France)
Abstract: In this contribution, we cast the challenging problem of continuous space Positron Emission Tomographic (PET)
spatial reconstruction in the context of point inverse problems. Namely, observations are discrete projections of detected
random emission locations whose probability distribution has to be estimated. The scanner’s limited field of view requires to
deal with truncated data, and scattered coincidences involves a mixture of projections sources. We follow a nonparametric
Bayesian approach where regularization of the inverse problem relies entirely on the nonparametric prior. We propose to
model the random distribution of recorded events emission locations using a Pitman-Yor Mixture of Normals prior (PYM)
and a Normal-Inverse Wishart model as base distribution for the Pitman-Yor Process. Thanks to a data augmentation
scheme, where emission locations for observed events are considered as hidden variables, we propose a hierarchical data
model. Though based on an exchangeable Polya urn representation, we develop a conditional sampler for PYM models using
an update formula from Pitman (1996) and a slice sampling strategy from recent work of Kalli, Griffin and Walker (revised,
2009). The MCMC algorithm is thus able to generate draws from the posterior distribution of the spatial intensity in order to
estimate desired functionals.
Co-authors: Mame Diarra Fall (L2S/SUPELEC, France), Claude Comtat (CEA-I2BM, France), Thomas Dautremer
(CEA-LIST, France), Thierry Montagu (CEA-LIST, France), Ali Mohammad Djafari (L2S/SUPELEC, France), Regine
Trebossen (CEA-I2BM, France)
Nonparametric Bayesian density estimation on manifolds with applications to planar shapes
Abhishek BHATTACHARYA (Duke University)
Abstract: Statistical analysis of landmark based shapes has diverse applications in morphometrics, medical diagnostics,
machine vision, robotics and many more. These shape spaces are non-Euclidean quotient manifolds, often the quotient of
the unit sphere under a group of transformations. To do nonparametric inference on them, one may define notions of center
and spread of a probability distribution on an arbitrary manifold and work with their estimates. Recently there has been a
significant amount of work done in this direction using Bootstrap and asymptotic methods. However in some applications,
these parameters may not be sufficient to identify the underlying probability. Then we need to estimate the probability
density itself by nonparametric methods. In this poster, I present a mixture model for the density with a suitable kernel on a
non-Euclidean manifold and obtain conditions under which the Kullback-Leibler property holds. Then I use Gibbs sampling
methods to obtain the Bayes estimate of the density. Similar results are obtained for the Planar Shape Space using a specific
kernel. The methods have been applied to a two sample data of shapes of 2D images to distinguish between the distribution
of the two groups.
Co-author: David Dunson (Duke University)
Bayesian local mixtures of factor analyzers
Anirban BHATTACHARYA (Duke University)
Abstract: Latent factor models are widely used for dimensionality reduction and sparse modeling of multivariate observations.
Mixtures of Factor Analyzers (MFA) provide a flexible generalization of Gaussian latent factor models. Dirichlet process
(DP) priors have been used to allow the number of components in MFA to be unknown and increasing with sample size.
However, such models make an implicit global clustering assumption, which can induce large numbers of clusters and hence
inefficiencies in high-dimensional applications. We provide background on DP-MFA models, and introduce a new local MFA
(L-MFA) framework. The proposed L-MFA places a local partition process (LPP) prior (Dunson, 2009, Biometrika) on a
mixture distribution for the mean and factor loadings parameters. This approach is shown to have substantial advantages
over existing MFA models. For posterior computation, we propose a parameter-expanded (PX) Gibbs sampling algorithm,
which avoids truncation of the infinite mixture distribution, while inducing a heavy-tailed default prior and ensuring efficient
mixing. The methods are illustrated with several examples.
28
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
How to use product partitions models to reflect the prior knowledge of the stratification in finite populationsampling
Eunice CAMPIRAN (UNAM Mexico)
Abstract: In Bayesian statistics, the prior distribution of the parameters reflects the initial beliefs of the researcher. After data
are observed, we have a learning process which is reflected in the posterior distribution of parameters. In finite population
sampling the researcher not only has prior knowledge of the parameters of interest, but also in the structure of the population.
In classical statistics, the experience of the researcher can help to divide the population into relatively homogeneous subgroups
in order to reduce the variability of the estimates of interest. We propose to use product partition models to model the
prior knowledge of the structure of the population and the parameters of interest. Using the ideas exposed in Quintana and
Iglesias (2003), we use a lost function that allows us to choose the stratification less expensive. We explore in which cases
this procedure is a learning process, and we can use the posterior distribution of the partitions to make stratification for
another survey related with the first one. Finally, we study a generalization of this model, when we do not assume a specific
distribution for each stratum, instead, we will use a random measure to make inference.
REFERENCES
Fernando A. Quintana and Pilar L. Iglesias. Bayesian clustering and product partition models. Journal of the Royal Statistical
Society: Series B (Statistical Methodology), 65(2), 557–574, 2003.
Modelling multi-output stochastic frontier using copulas
Alessandro CARTA (University of Warwick)
Abstract: The aim of this work is to introduce a new econometric methodology for multi-output production frontiers.
In the context of a system of frontier equations, we use a flexible multivariate distribution for the inefficiency error term.
This multivariate distribution is constructed through a copula function which allows for separate modelling of the marginal
inefficiency distributions and the dependence. We pay specific attention to the elicitation of a sensible (improper) prior
and provide a simple sufficient condition under which inference can be conducted. We develop an MCMC sampler. We
use Bayes factors to compare various copula specifications in the empirical context of Dutch dairy farm data, with two outputs.
Co-author: Mark Steel (University of Warwick)
29
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
On a class of Bayesian nonparametric priors derived by subordination of Stable processes
Annalisa CERQUETTI (Collegio Nuovo Pavia)
Abstract: Normalization of completely random measures has been widely studied in Bayesian nonparametrics in view to define
tractable alternatives to the Dirichlet process to be used as priors on the space of probability distributions. This construction
typically relies on an infinitely divisible r.v., striclty positive and almost surely finite, whose probability density is uniquely
identified by the corresponding Levy measure through the Levy-Khintchine representation of its Laplace transform. Until now
many different proposals have been derived by exploiting two well-known techniques to produce ID r.v.’s by a given one:
finite convolutions (Lijoi et al., 2005a), and exponential tilting, (Lijoi et al., 2005b; Cerquetti, 2007). We investigate a new
class of Bayesian nonparametric priors derived by convolution mixtures of the (positive) Stable r.v. by an independent ID r.v.
belonging to the family of Generalized Gamma convolutions (Bondesson, 1992). Relying on recent results for general BNP
priors obtained by normalization (James et al., 2008) we derive the corresponding predictive distributions and a comprehensive
posterior analysis. The specific form of the induced exchangeable partition probability function, which will be outside the
Gibbs class, will be also investigated in view of possible applications in a Bayesian nonparametric treatment of statistical
issues arising in genomic applications.
REFERENCES
Bondesson, L. (1992) Generalized Gamma convolutions and related classes of distributions and densities. Lecture Notes in
Statistics, 76, Springer-Verlag, New York.
Cerquetti, A. (2007) A note on Bayesian nonparametric priors derived from exponentially tilted Poisson-Kingman models.
Stat. Prob. Letters, 77, 1705-1711.
James, L.F., Lijoi, A. and Pruenster, I. (2008) Posterior analysis for Normalized Random Measures with Independent
Increments. Scand. J. Statist. (In press)
Lijoi, A., Mena, R.H. and Pruenster, I. (2005a) Bayesian nonparametric analysis for a generalized Dirichlet process prior.
Stat. Infer. Stoch. Proc. 8, 283-309.
Lijoi, A., Mena, R.H. and Pruenster, I. (2005b) Hierarchical mixture modelling with normalized inverse Gaussian priors. J.
Amer. Statist. Assoc. 100, 1278-1291.
Fast approximate Bayesian functional mixed effects model
James M. CIERA (University of Padua)
Abstract: Many medical studies collect functional data, such as trajectories in a biomarker over time. It is of interest
to estimate the trajectories and identify or predict clinically-important features. Linear mixed effects (LME) models are
commonly used in such cases, with non-linear effects easily incorporated through splines. However, for sufficient flexibility, it is
often necessary to use adaptive splines in which the number and locations of knots is unknown and potentially varying across
subjects. This can be accomplished with MCMC methodology, using reversible jump or stochastic search variable selection.
However, such approaches are infeasible to implement routinely, particularly for large data sets. Motivated by methods
proposed in the machine learning literature for compressive sensing, this paper proposes an approach for fast, approximate
Bayes functional data analysis relying on sparseness favouring hierarchical priors for basis coefficients. The proposed methods
are used to rapidly estimate individual-specific functions, while identifying features of interest. An application to analysis of
basal body temperature curves over the menstrual cycle is presented.
Co-authors: David B. Dunson (Duke University)
30
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Proxy Maximum Probability Estimation of Poisson Intensities
Jose C.S. DE MIRANDA (University of Sao Paulo)
Abstract: We propose a non parametric methodology of estimation of the intensity for Poisson point process on Rm.
We assume the observation region is a bounded Rm interval. The space of positive functions formed by composition of
L2(O)-functions with the exponential is endowed with a probability induced from another one defined on the set of wavelet
coefficients. This is a convenient space for the intensity to belong to and we choose as our first estimate for the intensity
a function that corresponds to the maximum posterior probability given a trajectory of the Poisson process on O. A second
estimate is obtained by suitably writing the posterior probability as a product of functions that are maximized separately
giving raise to proxy maximum posterior probability estimation. An adaptive thresholding procedure based on jointly testing
hypothesis on the wavelet coefficients, and adjusting priors’ locations is given.
On the robustness of Bayesian modelling of location and scale structures using heavy-tailed distributions
Chang C.Y. DOREA (University of Brasılia)
Abstract: For solving conflicting information between data and prior distributions Bayesian modelling with heavy-tailed
distributions is applied. Exploiting properties of regularly varying functions and distribution functions as well as their
relationship with the finiteness of the moments, we establish results for both location and shape structures. And, as a side
result, rates of convergence are derived.
Co-author: C.E. Guevara Otiniano (University of Brasılia)
Priors for vectors of survival functions
Ilenia EPIFANI (Polytechnic Milan)
Abstract: This poster describes a new nonparametric prior for a pair of dependent survival functions obtained by a suitable
transformation of bivariate completely random measures. The dependence between the two random survival functions is
obtained by a Levy copula. In a nonparametric Bayesian framework, the new approach can be used to model two–sample
survival data. A posterior characterization, conditionally on possibly right-censored data, is provided. Such new approach
yields a natural extension of the more familiar neutral to the right process of Doksum (1974) for single survival curve. As a
by-product of the analysis of this new prior, one finds out that the marginal distribution of a pair of observations from the
two samples coincides with the Marshall-Olkin or the Weibull distribution according to specific choices of the marginal Levy
measures.
31
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Bayesian nonparametric modeling of cross-section vs. LET for the prediction of on-orbit upset rate
Marian FARAH (University of California Santa Cruz)
Abstract: This work is concerned with the vulnerability of spaceborne microelectronic to single event upset (SEU), a change
of state caused by ions or electromagnetic radiation (e.g. solar wind) striking a sensitive node. To measure the susceptibility
of a semiconductor device to SEU, testing is conducted by exposing it to high-energy heavy ions or protons produced in a
particle accelerator (PA). The number of upsets depends on the linear energy transfer (LET) for silicon and the cross-section
(CS) of interaction. The prediction of the on-orbit upset rate, the main scientific goal of PA experiments, is made by
combining the CS vs. LET for the device with the model for the orbit-specific radiation environment. Typically, a Weibull
CDF is used to model the CS vs. LET curve, a choice that is purely conventional. We propose a DP-based nonparametric
approach to modeling the CS vs. LET curve, which allows the data to drive the shape of the CS-LET relationship, and can
thus result in more accurate predictive inference for the on-orbit upset rate. We illustrate the practical utility of the proposed
methodology with data from three PA experiments.
Co-authors: Athanasios Kottas (University of California Santa Cruz), Robin Morris (University of California Santa
Cruz)
Sharing features among dynamical systems with Beta processes
Emily B. FOX (Massachusetts Institute of Technology)
Abstract: Many nonlinear dynamical phenomena can be effectively modeled by a system that switches among a set of
conditionally linear dynamical modes. The switching vector autoregressive (VAR) process is one such model which has
proven effective in fields as diverse as econometrics and human motion capture. In many applications, one would like
to model dynamic behaviors which are shared among several related time series. The benefits of accurately transferring
knowledge between sequences are two-fold: it allows more accurate estimation of dynamic parameters, and can reveal
interesting relationships among time series. To this end, we have developed a nonparametric Bayesian approach based on
the beta process. Our method allows each time series to switch between an arbitrary number of dynamical modes, while
encouraging the reuse of behaviors exhibited by other time series. Integrating over the latent beta process random measure
results in a predictive distribution on assignments to dynamical regimes which is equivalent to the Indian buffet process of
Griffiths and Ghahramani. Our MCMC inference algorithm efficiently computes Metropolis-Hastings acceptance probabilities
for proposed assignments of behaviors to sequences via the sum-product algorithm, implemented on the finite switching
VAR process induced by the currently instantiated dynamical modes. We show promising results on several time series datasets.
Co-authors: Erik B. Sudderth (Brown University), Michael I. Jordan (University of California Berkeley), Alan S. Willsky (MIT)
32
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Nonparametric Bayesian regression for replicated categorical responses
Kassandra FRONCZYK (University of California Santa Cruz)
Abstract: We present a Bayesian nonparametric mixture modeling framework for repeated binary trials, or more generally,
replicated count responses. The proposed mixture model is built from a dependent Dirichlet process prior to provide flexibility
in the functional form of both the response distribution and the probability of success. The dependence of the mixing
distributions is governed by the level of a continuous covariate. Applications include traditional dose-response settings and
developmental toxicity studies, in which the main purpose is to determine the relationship between the level of exposure to a
toxic chemical and the probability of birth defects. The data from these experiments are complex and exhibit combinations
of zero and n-inflation, over- and underdispersion, and kurtosis, causing difficulties for standard parametric approaches. The
proposed modeling framework yields highly flexible inference for key risk assessment objectives, such as prediction at new
dose levels, and inversion for the dose level corresponding to a specified probability of malformation. We illustrate the model
through a study of the effects of 2,4,5-trichlorophenoxiacetic acid, including comparison with a parametric binomial-logistic
model and a semiparametric model based on a product of mixtures of Dirichlet processes prior.
Co-author: Athanasios Kottas (University of California Santa Cruz)
Clustering of many financial time series using nonparametric Bayesian technics
Dimitrios GIANNIKIS (Athens University of Economics and Business)
Abstract: We consider the estimation of a large number of financial time series. We suppose that the underlying pricing
model that explains the financial returns is a Multi-Factor model. In financial applications, the intercept- known as
alpha- plays a crucial role, as it is commonly used in order to evaluate the skill of the fund managers. Our interest lies
in the identification of common alphas of the univariate time series. To this end, we classify the series in an unknown
number of clusters. Within a cluster, the series share the same alpha. In order to estimate the alphas and the clusters
we adopt Bayesian non-parametric MCMC methods. In particular, we use the connection between Product Partition
Models and Dirichlet Process Mixture models (Qunintana et al. 2003) and using a Gibbs sampling sheme we estimate
the alphas and the remaining parameters of the model. With that procedure the alpha of a particular series takes with
some probability a value equal to the alpha of another series and with some probability a new value. Having simulated
the samples of the model parameters the next step is to find the optimal partition which better fits the data. The best
partition model is the one that minimizes a given loss function. We propose a clustering algorithm, which explores efficiently
the model space (i.e. different partition-models). We provide various simulation studies and an application to real financial data.
Co-authors: Petros Dellaportas (Athens University of Economics and Business), Ioannis D.Vrontos (Athens University
of Economics and Business)
Efficient sequential Monte Carlo for inference on Kingman’s coalescent
Dilan GORUR (University College London)
Abstract: Kingman’s coalescent is a prior over binary trees that was developed for studying the genealogy of a set of
haploid organisms. Several different techniques have been developed for inference on the coalescent. Most prominent
inference techniques on the coalescent rely on Monte Carlo algorithms. We propose a new sequential Monte Carlo (SMC)
algorithm for inference. We integrate out the mutations using belief propagation and therefore have the coalescence
times and coalescing pairs as the states of the Markov chain. This representation leads to a proposal distribution which
results in an efficient sampler. We compare our method to other inference techniques on short tandem repeat (STR) data
and show that our method is at least as efficient as the alternatives for univariate data and is more efficient for multivariate data.
Co-authors: Yee Whye Teh (University College London)
33
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Dependent mixtures of Dirichlet processes
Spyridon J. HATJISPYROS and Theodoros NICOLERIS (University of the Aegean)
Abstract: In this poster we present an approach to modeling dependent nonparametric random density functions, based on the
well known mixture of Dirichlet process model. The idea is to use a technique for constructing dependent random variables,
first used for dependent gamma random variables. While the methodology works for an arbitrary number of dependent random
densities, with each pair having their own dependent structure, we focus the mathematics and estimation algorithm on two
dependent random density functions. The extension to the more general case simply involves more complicated workings but
the same mathematics. Numerical examples are presented.
Co-authors: Stephen G. Walker (University of Kent)
Non-parametric hyper Markov priors
Daniel HEINZ (Carnegie Mellon University)
Abstract: Graphical models are used to describe the conditional indepen- dence relations in multivariate data. They have
been used for a variety of problems, including log-linear models, network analysis, graphical Gaussian models, and genetics.
A distribution that satisfies the conditional independence structure of a graph is Markov. A graphical model is a family
of distributions that is restricted to be Markov with respect to a certain graph. In a Bayesian problem, one may specify
a prior over the graphical model. Such a prior is called a hyper Markov law if the random marginals also satisfy the
independence constraints. Most previous work has concentrated on parametric families. We explore graphical models based
on a non-parametric family of distributions, developed from Dirichlet processes. We present an application of this theory
called a hyper Dirichlet mixture of Gaussians, which is both graphical and non-parametric.
Robust processes for latent variables in dynamical factor models
Ricardo HENAO (Technical University of Denmark)
Abstract: In a previous work, we introduced an algorithm to learn factor models and directed acyclic graphs (DAG) within
the same framework. Initially, we considered a heavy-tailed independent identically distribution for the factors and furthermore
extended the model to handle smoothness of the data by using instead a Gaussian process prior. The original algorithm has
three components, (i) inference of an identifiable Bayesian sparse factor model, (ii) stochastic search over variable and latent
factor orderings to produce a candidate set of variable permutations and (iii) inference of a Bayesian sparse DAG model. Here
we only focus on the prior distribution for the factors because we are still interested in handling smoothness of the data but
without reducing the performance of our algorithm due to the presence of outliers – which are very frequent in real situations.
For this purpose, we want to investigate the effect of allowing for a non-Gaussian process for the factors and to present some
comparative results both using artificial and real data.
Co-authors: Ole Winther
34
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Bayesian nonparametric modelling of covariance functions, with application to time series and spatialstatistics
Gudmund H. HERMANSEN (University of Oslo)
Abstract: There exist several parametric and nonparametric strategies for modeling and estimating the dependency structure,
or covariance function, for discrete stationary time series. For most of the parametric models one can, without too much
difficulty, also apply a Bayesian approach, but there is no obvious way to proceed for using Bayesian nonparametric techniques
to solve such problems.
Our approach is to use the Bayesian nonparametric principles to construct priors and derive posterior inference about the
unknown covariance or correlation function for discrete stationary Gaussian time series. It turns out that this is most easily
done by working with the spectral representation in the frequency domain. Posterior inference is then achieved through
posterior simulation (MCMC) and also through direct calculations by applying clever approximations to the multivariate
Gaussian Likelihood (types of Whittle approximations), where approximative asymptotic inference and Bernstein-von Mise
type of results are derived. The results will be illustrated through examples with both real and simulated data.
The ideas presented can easily be extended to continuous time processes and spatial process, then posterior inference can,
with some effort, be obtained through simulations techniques, but where both exact and approximate posterior inference will
become rather more complicated.
Co-author: Nils L. Hjort (University of Oslo)
Semiparametric Bayes local additive models for longitudinal data
Zhaowei HUA (University of North Carolina)
Abstract: In longitudinal data analysis, there is commonly interest in assessing the impact of a predictor on the time-varying
trajectory in a response variable. In such settings, it is important to account for heterogeneity in the shape of the trajectory
among subjects, while also allowing the impact of the predictor to vary for different sub-groups of subjects. We propose a
flexible semiparametric Bayes approach for addressing this problem relying on a local partition process prior, which allows
flexible local borrowing of information across subjects and sub-groups. Methods are developed for local hypothesis testing,
allowing for the identification of time windows across which a predictor has a significant impact, while inducing a multiplicity
adjustment in identifying sub-group specific effects. Posterior computation proceeds via an efficient MCMC algorithm that
relies on the exact block Gibbs sampler to avoid finite approximations to the nonparametric model. The methods are assessed
using simulation studies and applied to an EEG data set from the literature.
Co-author: David B. Dunson (Duke University)
A novel class of mixtures of multivariate Polya trees
Alejandro JARA (Universidad de Concepcion)
Abstract: We propose a novel mixture of multivariate Polya trees prior to define a flexible model for unknown distributions
centered at the multivariate normal distribution or family. Our proposal reduces the undesireable sensitivity to the choice of
the partitions associated with Polya tree constructions even when the Polya tree is centered around a specific distribution. In
order to reduce the impact of the partition on statistical inference, the mixture is defined with respect to the decomposition
of the centering covariance matrix using a Haar prior. We illustrate our approach in the context of multivariate density
estimation and hierarchical models.
Co-author: Timothy Hanson (University of Minnesota)
35
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Bayesian semiparametric stochastic volatility modeling
Mark J. JENSEN (Federal Reserve Bank of Atlanta)
Abstract: This paper extends the existing fully parametric Bayesian literature on stochastic volatility to allow for more
general return distributions. Instead of specifying a particular distribution for the return innovation, nonparametric Bayesian
methods are used to flexibly model the skewness and kurtosis of the distribution while the dynamics of volatility continue to
be modeled with a parametric structure. Our semiparametric Bayesian approach provides a full characterization of parametric
and distributional uncertainty. A Markov chain Monte Carlo sampling approach to estimation is presented with theoretical
and computational issues for simulation from the posterior predictive distributions. The new model is assessed based on
simulation evidence, an empirical example, and comparison to parametric models.
Co-author: John Maheu (University of Toronto)
Mixtures of stick-breaking processes
Maria KALLI (University of Kent)
Abstract: We consider mixtures of stick–breaking processes as a generalization of the mixture of Dirichlet process model.
We focus our attention on the weights of these mixtures and their effect on the number of modes of the marginal density. We
provide specific reasons for using particular choices of prior for these weights. The sampler introduced in Kalli et al (2009) is
used to facilitate the density estimation. Numerical illustrations involving real data sets are presented.
Co-authors: Stephen G. Walker (University of Kent)
A Bayesian nonparametric causal model
George KARABATSOS (University of Illinois-Chicago)
Abstract: Typically, in the practice of causal inference from observational studies, a parametric model is assumed for the joint
population density of potential outcomes and treatment assignments, and possibly this is accompanied by the assumption of
no hidden bias. However, both assumptions are questionable for real data, the accuracy of causal inference is compromised
when the data violates either assumption, and the parametric assumption precludes capturing a more general range of density
shapes (e.g., heavier tail behavior and possible multi-modalities). We introduce a flexible, Bayesian nonparametric causal
model to provide more accurate causal inferences. The model makes use of a stick-breaking prior, which has the flexibility to
capture any multi-modalities, skewness and heavier tail behavior in this joint population density, while accounting for hidden
bias. We prove the asymptotic consistency of the posterior distribution of the model, and illustrate our causal model through
the analysis of small and large observational data sets.
Co-authors: Stephen G. Walker (University of Kent)
36
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Bayesian subspace clustering with Indian buffet process
Dohyun KIM (Seoul National University)
Abstract: Classical clustering algorithms consider all dimensions of data to measure the distance between samples in data.
One possible problem is existing noisy variable that is not informative to discriminate data. Variable selection procedures
in clustering algorithms can improve clustering quality by removing noisy variables. In many cases, noisy variables can be
different among pairs of groups and variable selection procedures in clustering algorithms cannot identify the set of these
noisy variables. Subspace clustering algorithms discover the set of noisy variables for each pair of groups and establish a
group structure. The Indian buffet process (IBP) is a Bayesian nonparametric distribution over a binary infinite matrix, which
entry of 1 in a binary infinite matrix indicates that a particular variable relates a latent group. With Indian buffet process,
we can represent that subsets of variables are used for given groups. We propose a Bayesian subspace clustering model with
mixture model with unknown components and IBP. We describe the Reversible Jump Markov Chain Monte Carlo(RJMCMC)
for posterior inference and investigate the performance of the proposed model by simulated and real data.
Co-authors: Yongdai Kim (Seoul National University)
The time varying survival analysis by Bayesian bootstrap
Gwangsu KIM (Seoul National University)
Abstract: We propose a Bayesian approach for time varying coefficient models in survival analysis. As a prior for a time
varying coefficient, we use a difference of two nondecreasing Levy processes such as gamma or beta processes which covers
all of bounded variation functions. As a prior for the baseline hazard function, we use the Bayesian bootstrap prior proposed
by Kim and Lee (2003) for easy computation. We develop an efficient MCMC algorithm.
Co-authors: Yongdai Kim (Seoul National University)
Bayesian nonparametric modelling of grouped data with an application to stochastic frontiers
Michalis KOLOSSIATIS (University of Warwick)
Abstract: Bayesian nonparametric methods offer a natural way of flexibly modelling various data sets. Most of these
models make use of the Dirichlet Process (DP), usually incorporated in some hierarchical setting. Alternative choices
include the Normalized Inverse-Gaussian Process, or more generally, the class of Normalized Random Measures. The
latter are created by normalising other random measures (for example, the DP, which is a normalised Gamma Process).
The idea here is to normalise infinitely divisible random measures, in order to create identically distributed, but not
independent random probability measures. By normalising two Gamma Processes, for example, we can create a model
similar to the one of Muller, Quintana and Rosner (2004), but with marginal distributions of the data in both groups
that follow a DP. Posterior inference for our proposed model is straightforward using Monte Carlo Markov Chain (MCMC)
methods, and a novel split/merge step, which improves mixing, is introduced. Additionally, the proposed model can be
naturally extended to cases of more than two correlated distributions. Finally, we incorporate our model in a Stochastic
Frontier hierarchical setting and analyse the efficiency of some hospitals, which are grouped according to discrete characteristics.
Co-authors: Mark Steel (University of Warwick), Jim Griffin (University of Kent)
37
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Adaptive Bayesian density estimation with location-scale mixtures
Willem KRUIJER (Universite Paris Dauphine)
Abstract: We study convergence rates of Bayesian density estimators based on finite location-scale mixtures of a kernel that is
assumed to be in a class containing the normal and Laplace density. We construct a finite mixture approximation of densities
whose logarithm is locally beta-Holder, with squared integrable Holder constant. Under additional tail and moment conditions,
the approximation is minimax for both the supremum-norm and the Kullback-Leibler divergence. We use this approximation to
establish convergence rates for a Bayesian mixture model with priors on the weights, locations, and the number of components.
locally Holder continuous. Regarding these priors, we provide general conditions under which the posterior converges at
a near optimal rate, and is rate-adaptive with respect to the smoothness of the log of the true density. Examples of pri-
ors which satisfy these conditions include Dirichlet and Polya-tree priors for the weights, and Poisson processes for the locations.
Co-authors: Judith Rousseau (Universite Paris Dauphine and CREST), Aad van der Vaart (VU University Amster-
dam)
Posterior convergence and model estimation in Bayesian change-point problems
Heng LIAN (Nanyang Technological University)
Abstract: We study the posterior distribution of the Bayesian multiple change-point regression problem when the number and
the locations of the change-points are unknown. While it is relatively easy to apply the general theory to obtain the O(1/√n)
rate up to some logarithmic factor, showing the exact parametric rate of convergence of the posterior distribution requires
additional work and assumptions. Additionally, we demonstrate the asymptotic normality of the segment levels under these
assumptions. For inferences on the number of change-points, we show that the Bayesian approach can produce a consistent
posterior estimate. Finally, we argue that the point-wise posterior convergence property as demonstrated might have bad
finite sample performance in that consistent posterior for model selection necessarily implies the maximal squared risk will be
asymptotically larger than the optimal O(1/√n) rate. This is the Bayesian version of the same phenomenon that has been
noted and studied by other authors.
Large deviations for Bayesian estimators in first-order autoregressive processes
Claudio MACCI (University of Rome Tor Vergata)
Abstract: In this paper we consider first-order autoregressive processes and we allow either centered Normal or exponential
innovations. We prove large deviation principles for posterior distributions on the unknown parameter and, motivated by
potential applications in risk theory, we also prove large deviation principles for Bayesian estimators of the Lundberg’s parameter.
An alternative nonparametric time-series model
Juan-Carlos MARTINEZ-OVANDO (University of Kent)
Abstract: Bayesian nonparametric modelling of dynamic processes has received considerable attention in recent years. In this
poster we propose a stationary state-space model provided with flexible (nonparametric) dependent and marginal structures.
We use the above construction to define a dynamic mean-variance mixture model with nonparametric margins. We briefly
sketch a MCMC sampling algorithm and present an example to show the flexibility of the model in describing dependence of
financial returns.
Co-authors: Stephen G. Walker (University of Kent)
38
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Dirichlet process EM algorithm for semi-supervised learning
Takashi MATSUMOTO (Waseda University)
Abstract:Learning a dataset containing labeled data and unlabeled data is known as the semi- supervised learning problem
[1][2] among others. These problems can be found in many fields, including signal processing, image processing, pattern
recognition, and machine learning. This work attempts to propose a semi-supervised learning scheme using a Bayesian
Maximum A Posteriori Expectation Maximization (MAP-EM) algorithm with Dirichlet process prior. With stick-breaking
representation [3] one can explicitly derive MAP-EM in both batch and recursive manners. This enables us to estimate a
mixture model with an unknown number of components. It also provides a simpler implementation for a mixture model with
a stick- breaking process prior than other implementations such as Markov Chain Monte Carlo (MCMC). Two examples are
examined to validate the proposed scheme. One of the examples tested against is the dual shrinking spirals data in [4]. The
proposed algorithm appears functional. The poster also attempts an automatic adjustment of the hyperparameter associated
with the base distribution of the Dirichlet process prior.
REFERENCES
[1] V. K. Mansinghka et. al., AISTATS, 2007.
[2] B. Krishnapuram et. al., NIPS, 2004.
[3] J. Sethuraman, Stat. Sin., 1994.
[4] N. Ueda et. al. Neural Comput., 2000.
Co-authors: T. Kimura (Waseda University), Y. Nakada (Waseda University), T. Tokuda (Waseda University)
Bayesian modeling of joint and conditional distributions
Andriy NORETS (Princeton University)
Abstract: We propose a Bayesian approach to flexible modeling of conditional distributions. The approach uses a flexible
model for the joint distribution of the dependent and independent variables and then extracts the conditional distributions
of interest from the estimated joint distribution. We use a finite mixture of multivariate normals to estimate the joint
distribution. The conditional distributions can then be assessed analytically or through simulations. The discrete variables are
handled through the use of latent variables. The estimation procedure employs an MCMC algorithm. We provide a frequentist
justification of the method: a Bayesian estimator of the density is consistent in the total variation distance. In contrast to
previous theoretical research on finite mixtures of normals, we consider multivariate case, discrete variables, and priors that are
actually used in practice. Also, we explicitly characterize the class of densities that can be consistently estimated. Experiments
demonstrate that the method can be used as a heteroscedasticity and non-linearity robust regression model with discrete and
continuous dependent and independent variables and as a Bayesian alternative to semi- and non-parametric models such as
quantile and kernel regression.
Co-authors: Justinas Pelenis (Princeton University)
39
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Bayesian nonparametric regression with an unknown predictor-dependent residual distribution
Debdeep PATI (Duke University)
Abstract: We consider the problem of estimation and Bayesian inferences on E(y|x), allowing the residual density to be
changing flexibly with predictors x, while enforcing the unimodal and symmetric about zero constraint. The proposed model is
based on a Gaussian process prior for the mean regression function, E(y|x), and a probit stick-breaking (PSBP) scale mixture
of Gaussian prior for the collection of residual densities indexed by predictors x. Initially considering the special case in which
the residual density is homoscedastic, we provide fairly general sufficient conditions to ensure weak posterior consistency in
estimating both the mean regression function and the residual density. This generalizes existing theory focusing on Gaussian
residual distributions. By using a scale mixture of Gaussians that changes adaptively with predictors, we obtain a robust
Bayesian regression procedure that automatically down weighs outliers and influential observations. Posterior computation
relies on an efficient data augmentation exact blocked Gibbs sampler, which avoids truncation in modeling the collection of
unknown residual distributions. The methods are illustrated using simulated and real data applications.
A Bayesian methodology for models adequacy
Maria JOAO POLIDORO (Polytechnic Porto)
Abstract: An important issue in statistical modeling is to evaluate the fitting of a proposed parametric model to a
given dataset. The Bayesian foundation for fit evaluation is conveyed by the predictive distribution of the observables
through exploratory methods, or by using formal posterior predictive model checks, like Bayesian p-values. These methods
compare observed values with predicted values. Discrepancies between them are a symptom that the data may not been
generated by the proposed model. An enhanced alternative to such methods consists on embedding the proposed model
in a nonparametric model. Then, to validate the proposed model, the parametric fit is compared with the nonparametric
using proper summaries of comparison (e.g., Bayes factor). The most common approach to achieve the nonparametric
model uses Dirichlet process priors. Departing from a consistent vision about the Bayesian nonparametric statistics and
model adequacy state-of-art we use simulation techniques to compare the different approaches and explore further alternatives.
Co-authors: Fernando Magalhaes (Polytechnic of Porto), Maria Antonia Amaral Turkman (University of Lisbon)
Countable mixtures of Markov chains
Cecilia PROSDOCIMI (University of Padua)
Abstract: In 1980 Diaconis and Freedman proposed an extension of de Finetti’s characterization theorem to partially
exchangeable sequences, proving that a recurrent sequence is partially exchangeable if and only if it is a mixture of Markov
Chains(MC-s). We concentrate our attention on the special class of partially exchangeable sequences that are a countable
mixtures of MC-s, that is mixtures where the prior distribution on the mixed MC-s is concentrated on a countable set. We
show that a partially exchangeable sequence is a countable mixture of MC-s if and only if it is a Hidden Markov Model (HMM)
with a countable state space of the underlying Markov chain. Our main theorem is an extension, to partially exchangeable
sequences, of an old result of Dharmadhikari (1964) for exchangeable sequences. Many interesting statistical problems for
mixtures of MC-s can be investigated, from the inference on the mixture component measures, to the estimation of the
memory of the MC-s being mixed (see Quintana et al. 1998). It seems to us interesting to deepen the understanding of
the connection between models widely used in applications, as HMM-s and mixtures of MC-s, as this could lead to new
approaches to solve inference problems for both classes.
Co-authors: Lorenzo Finesso (CNR Italy)
40
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Semiparametric Bayesian approach to gene profile classification
Sandra RAMOS (High Institute of Engineering of Oporto)
Abstract: Screening methods are based on the information of a feature vector X from an individual to decide if further
investigation is worth to check whether an attribute is present (variable Y belongs to a known region Cy), in which case
he/she is called a ”success”. Screening makes sense when it is difficult/expensive to observe Y, but X is both accessible
and informative about Y. The issue reduces to the problem of finding a region Cx for X, such that if X belongs to Cx, the
probability of success is raised (Boys and Dunsmore, 1986). Assuming a parametric Bayesian framework, an optimal region
Cx contains the values x for which the Bayesian predictive probability of success conditional on x is above a certain threshold
(Turkman and Amaral Turkman,1989). We aim to develop screening procedures relaxing the parametric assumption. As
a first approach, a semiparametric Bayesian solution is obtained by using kernel smoothing techniques to estimate the
conditional distribution of X given Y=y. We demonstrate its usefulness when applied to pairs of gene expression levels for
binary classification purposes, using DNA microarray data and compare it with a parametric approach.
REFERENCES
Boys,R.J. and Dunsmore,I.R., 1986. Screening in a normal model.JRSS, B48, 60-69. Turkman,K.F. and Amaral Turk-
man,M.A., 1989. Optimal Screening Methods. JRSS, B51, 287-295.
Co-authors: Antonia Amaral Turkman (University of Lisbon), Marılia Antunes (University of Lisbon)
Causal analysis with chain event graphs
Eva RICCOMAGNO (University of Genoa)
Abstract: The chain event graph (CEG) is a graphical model specifically designed for the representation and analysis
of discrete asymmetric problems. We present a causal extension to the CEG, analogous to the extension of Bayesian
Networks to Causal Bayesian Networks. We show that the analysis of causal effects can be performed through examination
of the topology of the CEG and use this idea to develop a back-door theorem for CEGs. First we show how we gener-
alised the notion of causality for Bayesian network in Pearl (2000) coupled with work in Shafer (1996) to derive the causal CEG.
Co-authors: Jim Q. Smith (University of Warwick), Peter Thwaites (University of Warwick)
Nested partition models
Abel RODRIGUEZ (University of California at Santa Cruz)
Abstract: In this work we consider a flexible model for matrix-variate data based on the two parameter Poisson-Dirichlet
process. The model is motivated by a study of cancer mortality rates in the U.S., where rates for different types of cancer
are available for each state. In this setting, we are interested in improving estimation by flexibly borrowing across states and
cancers with similar outcomes. The resulting model allows us to cluster states with similar mortality rates across all cancers
while also allowing us to identify, within each cluster of states, cancers with similar outcomes.
Co-authors: Kaushik Ghosh (University of Nevada)
41
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Sampling the mixture normal model
Carlos RODRIGUEZ (University of Kent)
Abstract: We describe an MCMC method to sample the posterior distribution of a Mixture Normal Model with unknown
number of components. Using ”standard” Metropolis Hastings and Gibbs sampler ideas from Tierney (1994) we are able to
sample from the joint posterior distribution of all variables. We compare with the Reversible Jump methodology of Richardson
and Green (1997).
Co-authors: Stephen G. Walker (University of Kent)
On the computability of de Finetti measures
Daniel M. ROY (Massachusetts Institute of Technology)
Abstract: The complexity of probabilistic models, especially those involving recursion and nonparametrics, has far exceeded
the representational capacity of graphical models. Functional programming languages with probabilistic choice operators have
recently been proposed as universal representations for statistical modeling (e.g., IBAL (Pfeffer01), lambda (ParkEtAl2008),
Church (GoodmanEtAl2008).
Although the semantics of probabilistic programs have been studied extensively in theoretical computer science in the context
of randomized algorithms, this application to universal statistical modeling has a different character which raises interesting
theoretical questions. Here we describe a recent result on computability, exchangeability and de Finetti measures, and
highlight its consequences for the semantics of probabilistic programs and for statistical inference.
We prove a uniformly computable version of de Finetti’s theorem. The classical result states that an exchangeable sequence
is a mixture of i.i.d. sequences. Moreover, there is a measure-valued random variable, the directing random measure, which
renders the sequence i.i.d. The distribution of the directing random measure is called the de Finetti measure. We show that
computable exchangeable sequences of real random variables have computable de Finetti measures.
This result implies that it is possible to systematically remove mutation from probabilistic programs. These transforms expose
the conditional independence of the underlying process and enable efficient, parallel inference.
Co-author: Cameron Freer (MIT)
Gibbs sampling the two-parameter Poisson-Dirichlet process and its diffusion limit
Matteo RUGGIERO (University of Pavia)
Abstract: We study a simple MCMC path based on the Pitman urn scheme and show that under certain conditions its limit
is an infinite dimensional diffusion on the simplex which is stationary with respect to the two-parameter Poisson-Dirichlet
distribution. The limiting diffusion generalises the classical infinitely-many-neutral-alleles model in population genetics.
Co-author: Stephen G. Walker (University of Kent)
42
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Rates of convergence for Bayes and ML estimators of mixtures of exponential power densities
Catia SCRICCIOLO (Bocconi University)
Abstract: In this work we consider estimating densities that are location or location-scale mixtures of kernels in the family
of exponential power distributions which includes the Laplace and normal distributions as special cases. Posterior rates of
convergence for finite location-scale mixtures of exponential power densities have been considered by Kruijer (2008). We
focus on rates of convergence in Hellinger distance for ML and Bayes estimators. This problem has been studied by Ghosal
and van der Vaart (2001) for sampling densities that are location or location-scale mixtures of normals with a compact
set of locations or with the true mixing distribution having sub-Gaussian tails, under the severe assumption that the scale
parameter stays bounded away from zero and infinity. We address true densities that are location or location-scale mixtures
of exponential power densities without any assumption on the scale. Clearly, location and location-scale mixtures of normals
are covered as special cases. We show that ML estimators converge at near parametric rate except for a logarithmic term.
Defined a Dirichlet mixture of exponential power distributions as a prior on the target class of densities, the posterior is shown
to converge also at near parametric rate, the power of the log-term depending on the tail behavior of the base measure and
the prior on the scale parameter in the case of location mixtures, and on the tail behavior of the base measure of the Dirichlet
process for the overall mixing parameter in the case of location-scale mixtures.
REFERENCES
Ghosal, S. and van der Vaart A. W. (2001) Entropies and rates of convergence for maximum likelihood and Bayes estimation
for mixtures of normal densities, Annals of Statistics, Vol. 29, No. 5, 1233-1263.
Kruijer W. (2008) Convergence Rates in Nonparametric Bayesian Density Estimation. PhD-thesis. Department of Mathemat-
ics, Vrije Universiteit Amsterdam, http://www.math.vu.nl/ kruijer/PhDthesis Kruijer.pdf, 2008.
Genealogy and dual processes associated with time-dependent Dirichlet processes, and their polynomialeigenfunctions
Dario SPANO (University of Warwick)
Abstract: Two classes of measure-valued processes with Ferguson-Dirichlet stationary measure and orthogonal polynomial
eigenfunctions are examined: none of them is a diffusion, one of them is not even Markov. Several properties of their
evolution, such as the convergence to stationarity is studied by looking at the property of their dual genealogical process.
Known results on orthogonal polynomials are reinterpreted under a probabilistic approach.
Conditional modeling for survival data
Matthew TADDY (University of Chicago Booth School of Business)
Abstract: We develop a fully nonparametric framework for analysis of multivariate survival data in the presence of a regression
component. The approach is unified in that all functionals of interest are available through a single inferential process for
conditional response densities, based on Dirichlet process mixture prior probability models for the joint distribution of survival
responses and covariates. This model is able to handle a variety of types of censoring in the response and provides a natural
framework for inference about multivariate responses. We have also extended the methodology to treatment-control settings,
through the use of dependent Dirichlet processes.
Co-authors: Athanasios Kottas (University of California Santa Cruz)
43
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Bayesian nonparametric model for integrating genomic data sets
Mahlet G. TADESSE (Georgetown University)
Abstract: In recent years, there has been a growing effort in integrating and analyzing various genomic data sets. For
example, in the context of transcriptomics and genome-wide association studies, a common goal is to identify groups of
correlated gene expression levels that are modulated by sets of DNA polymorphisms. We propose a Bayesian nonparametric
model to identify important features and relationships between data sets collected using various high-throughput technologies.
The proposed method combines the ideas of infinite mixture models, nonparametric regression, and variable selection to
formulate a flexible model that provides a unified approach for uncovering cluster structures in each data set and identify
groups of associated markers across data sets.
Maximum a posteriori estimation for Dirichlet process language models
Takaaki TOKUDA (Waseda University)
Abstract: In recent years, estimation of mixture distributions with Dirichlet Process (DP) prior have been successfully
applied to practical problems. MCMC [1] as well as Variational Bayesian (VB)[2] algorithms have been used for implementing
estimation of the parameters associated with the mixture distributions. This poster attempts to perform MAP Expectation-
Maximization with DP prior for language models. With stick breaking representation for DP [3], one can explicitly derive an
EM scheme. Our specific model consists of mixtures of unigrams and Polya mixtures. The algorithm is first tested against the
text data collected from the Cranfield corpus used in probabilistic Latent Semantic Indexing [4] which is a popular data set to
validate language models. The proposed algorithm is also tested against data collected from some Japanese Web sites. The
proposed algorithm appears to be working. Our future work includes comparison with other language models and automatic
adjustment of the hyperparameter associated with the DP prior.
REFERENCES
[1] R.M.Neal, Journal of Computational and Graphical Statistics, vol.9, pp.249-265,2000.
[2] D. Blei and M. I. Jordan, Journal of Bayesian Analysis, 1(1):121-144,2005.
[3] J. Sethuraman, Stat. Sin., vol. 4, pp. 639-650, 1994.
[4] T.Hofmann, SIGIR-99, pages 50-57, 1999.
Co-authors: T. Kimura (Waseda University), Y. Nakada (Waseda University), T. Matsumoto (Waseda University)
A sparse, infinite topic model based on the Indian Buffet Process
Sinead WILLIAMSON (University of Cambridge)
Abstract: Topic models such as Latent Dirichlet Allocation have proven very popular for modeling corpora of documents.
Non-parametric implementations of such topic models have allowed us to incorporate hierarchical structure into our topic
distribution and removed the need to specify the number of topics. Existing nonparametric approaches however do not
explicitly encapsulate the observation that, while the total number of latent topics may be countably infinite, documents
are typically composed from a small finite number of topics. In addition, the number of topics may be expected to grow in
expectation with the length of the document. We present a fully generative sparse non-parametric topic model based on the
Indian Buffet Process, and describe an efficient Gibbs sampler for inference in this model.
Co-authors: Chong Wang (Princeton University), Katherine Heller (University of Cambridge)
44
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
Bayesian consistency for regression model
Fei XIANG (University of Kent)
Abstract: We study Bayesian consistency for regression in both the weak and strong sense.
We show that if the prior distribution assigns positive probabilities to the Kullback-Leibler neighbourhoods of the true
distribution, then the posterior distribution is consistent in the weak topology. This result follows from a generalization of
a result of Schwartz to independent, non-identical case, and the existence of exponentially consistent test is replaced by a
requirement of the summability of the square root of the prior mass on certain neighbourhood of the true density, which is
straightforward in the weak case.
In the strong case, the positive prior mass on Kullback-Leibler neighbourhood is also needed, while the summability of the
square root of the prior mass on Hellinger neighbourhoods needs care.
We consider both a Normal and Bernoulli to illustrate the findings.
Co-authors: Stephen G. Walker (University of Kent)
Artificial Conditional Dirichlet Processes
Hao WU (Free University of Berlin)
Abstract: Recently, Dirichlet process (DP) has been widely applied to Bayesian nonparametric modeling for hidden Markov
models (HMMs) and jump Markov systems (JMSs), which allows potentially infinite number of hidden states. Nonetheless,
the original DP model can only describe the exchangeable processes. And some improved approaches, such as hierarchical
Dirichlet process (HDP), involve very complicated prior models. In this poster, we propose a new artificial conditional Dirichlet
process (ACDP) to approximate the Markov dynamics of hidden states. This model uses DP to design the prior distribution
of transition pair of hidden states instead of single states, and satisfies the requirement of time dependency by introducing
artificial constraints. In comparison with other DP based models, ACDP needs only a few instrumental variables and is easier
to be estimated through Bayesian sampling.
Mean field inference for the Dirichlet process mixture model
Oliver ZOBAY (University of Bristol)
Abstract: We present a systematic study of several recently proposed methods of mean field inference for the Dirichlet
process mixture (DPM) model that are based on the truncated stick-breaking representation and related approaches [1,2].
These methods provide approximations to the exact posterior distribution. We investigate density estimation and cluster
allocation and compare to exact results. Further, more specific topics include the general mathematical structure of the
mean field approximation, the handling of the truncation level, the effect of including a prior on the concentration parameter
of the DPM model, the relationship between the proposed variants of the mean field approximation, and the connection to
maximum a-posteriori estimation of the DPM.
45
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
List of Participants 1/3
Full Name Affiliation E-mail
Isadora Antoniano Villalobos University of Kent, UK [email protected]
Julyan Arbel INSEE-ENSAE-CREST, France [email protected]
Raffaele Argiento CNR-IMATI, Italy [email protected]
Eric Barat CEA LIST, France [email protected]
Federico Bassetti University of Pavia, Italy [email protected]
Abhishek Bhattacharya Duke University, USA [email protected]
Anirban Bhattacharya Duke University, USA [email protected]
Pier Giovanni Bissiri University of Cagliari, Italy [email protected]
Natalia Bochkina University of Edinburgh, UK [email protected]
Eunice Campiran Universidad Nacional Autonoma de Mexico, Mexico eunice [email protected]
Francois Caron University of British Columbia, Canada [email protected]
Alessandro Carta University of Warwick, UK [email protected]
Ismael Castillo Vrije Universiteit Amsterdam, Netherlands [email protected]
Annalisa Cerquetti Collegio Nuovo, Pavia, Italy [email protected]
James M. Ciera University of Padova, Italy [email protected]
Enkeleda Cuko SUNY at Albany, USA [email protected]
Pierpaolo De Blasi University of Turin, Italy [email protected]
Maria De Iorio Imperial College, UK [email protected]
Jose C.S. de Miranda University of Sao Paulo, Brazil [email protected]
Dipak K. Dey University of Connecticut, USA [email protected]
Persi Diaconis Stanford University, USA [email protected]
Emanuele Dolera University of Pavia, Italy [email protected]
Chang Dorea Universidade de Brasilia, Brazil [email protected]
Arnaud Doucet University of British Columbia, Canada [email protected]
David B. Dunson Duke University, USA [email protected]
Omar El Dakkak University of Paris VI, France [email protected]
Ilenia Epifani Politecnico di Milano, Italy [email protected]
Michael Escobar University of Toronto, Canada [email protected]
Stefano Favaro University of Torino, Italy [email protected]
Mame Diarra Fall L2S SUPELEC, France [email protected]
Marian Farah University of California Santa Cruz, USA [email protected]
Giorgio Ferrari University of Rome La Sapienza, Italy [email protected]
Emily B. Fox Massachusetts Institute of Technology, USA [email protected]
Kassandra Fronczyk University of California Santa Cruz, USA [email protected]
Ursula Garczarek Unilever R&D Vlaardingen, Netherlands [email protected]
Mauro Gasparini Politecnico di Torino, Italy [email protected]
Alan Gelfand Duke University, USA [email protected]
Zoubin Ghahramani University of Cambridge, UK [email protected]
Subhashis Ghosal North Carolina State University, USA [email protected]
Dimitrios Giannikis Athens University of Economics and Business, Greece [email protected]
Dilan Gorur University College London, UK [email protected]
Peter Green University of Bristol, UK [email protected]
Jim Griffin University of Kent, UK [email protected]
Alessandra Guglielmi Politecnico di Milano, Italy [email protected]
Michele Guindani University of New Mexico, USA [email protected]
Georgia Hadjicharalambous University of Turin, Italy [email protected]
Tim Hanson University of Minnesota, USA [email protected]
Spyridon Hatjispyros University of the Aegean, Greece [email protected]
Daniel Heinz Carnegie Mellon University, USA [email protected]
Ricardo Henao Technical University of Denmark, Denmark [email protected]
Gudmund Hermansen University of Oslo, Norway [email protected]
Amy H. Herring University of North Carolina at Chapel Hill [email protected]
Nils L. Hjort University of Oslo, Norway [email protected]
46
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
List of Participants 2/3
Full Name Affiliation E-mail
Peter Hoff University of Washington, USA [email protected]
Chris C. Holmes University of Oxford, UK [email protected]
Susan Holmes Stanford University, USA [email protected]
Silvano Holzer University of Trieste, Italy [email protected]
Zhaowei Hua University of North Carolina, USA [email protected]
Sam Hui New York University, USA [email protected]
Rosalba Ignaccolo University of Turin, Italy [email protected]
Daniele Imparato Politecnico di Torino, Italy [email protected]
Lancelot F. James Hong Kong University of Science and Technology, HK [email protected]
Alejandro Jara Universidad de Concepcion, Chile [email protected]
Mark J. Jensen Federal Reserve Bank of Atlanta, USA [email protected]
Michael I. Jordan University of California Berkeley, USA [email protected]
Arbel Julyan INSEE - ENSAE - CREST, France [email protected]
Maria Kalli University of Kent, UK [email protected]
George Karabatsos University of Illinois-Chicago, USA [email protected]
Dohyun Kim Seoul National University, South Korea [email protected]
Gwangsu Kim Seoul National University, South Korea [email protected]
Yongdai Kim Seoul National University, South Korea [email protected]
Bas Kleijn University of Amsterdam, Netherlands [email protected]
Bartek Knapik VU University Amsterdam, Netherlands [email protected]
Michalis Kolossiatis University of Warwick, UK [email protected]
Willem Kruijer University of Paris Dauphine, France [email protected]
Lucia Ladelli Politecnico di Milano, Italy [email protected]
Jaeyong Lee Seoul National University, South Korea [email protected]
Heng Lian Nanyang Technological University, Singapore [email protected]
Antonio Lijoi University of Pavia, Italy [email protected]
Albert Lo Hong Kong University of Science and Technology, HK [email protected]
Claudio Macci Universita di Roma Tor Vergata, Italy [email protected]
Steven N. MacEachern Ohio State University, USA [email protected]
Andrea Martinelli University of Insubria, Italy [email protected]
Juan-Carlos Martinez-Ovando University of Kent, UK [email protected]
Takashi Matsumoto Waseda University, Japan [email protected]
Karla Medina University of Turin, Italy [email protected]
Silvia Montagna University of Torino, Italy [email protected]
Pietro Muliere Bocconi University, Italy [email protected]
Peter Muller University of Texas, USA [email protected]
Theodoros Nicoleris University of the Aegean, Greece [email protected]
Consuelo Nava University of Torino, Italy [email protected]
Luis E. Nieto-Barajas ITAM, Mexico [email protected]
Bernardo Nipoti University of Pavia, Italy [email protected]
Takeshi Nokajima Waseda university, Japan [email protected]
Andriy Norets Princeton University, USA [email protected]
Andrea Ongaro Universita di Milano-Bicocca, Italy [email protected]
Peter Orbanz University of Cambridge, UK [email protected]
Antonio A. Ortiz Barranon University of Kent, UK [email protected]
Kosuke Ota Waseda University, Japan [email protected]
Omiros Papaspiliopoulos Pompeu Fabra University, Spain [email protected]
Debdeep Pati Duke University, USA [email protected]
Giovanni Peccati Universite Paris Ouest Nanterre, France [email protected]
Sonia Petrone Bocconi University, Italy [email protected]
Giovanni Pistone Politecnico di Torino, Italy [email protected]
Maria Joao Polidoro Universidade de Lisboa, Portugal [email protected]
Cecilia Prosdocimi University of Padova, Italy [email protected]
Igor Prunster University of Torino, Italy [email protected]
47
Moncalieri, June 21–25, 2009 7th Workshop on Bayesian Nonparametrics
List of Participants 3/3
Full Name Affiliation E-mail
Anthony P. Quinn Trinity College Dublin, Ireland [email protected]
Fernando A. Quintana Pontifical Catholic University of Chile, Chile [email protected]
RV Ramamoorthi Michigan State University, USA [email protected]
Sandra Ramos High Institute of Engineering of Oporto, Portugal [email protected]
Eugenio Regazzini University of Pavia, Italy [email protected]
Eva Riccomagno Universita di Genova, Italy [email protected]
Gareth Roberts University of Warwick, UK [email protected]
Abel Rodriguez University of California at Santa Cruz, USA [email protected]
Carlos E. Rodriguez University of Kent [email protected]
Alex Rojas Carnegie Mellon University in Qatar, Qatar [email protected]
Judith Rousseau University of Paris Dauphine, France [email protected]
Daniel M. Roy Massachusetts Institute of Technology, USA [email protected]
Matteo Ruggiero University of Pavia, Italy [email protected]
Chiara Sabatti UCLA, USA [email protected]
Marina Santacroce Politecnico di Torino, Italy [email protected]
Bruno Scarpa University of Padova, Italy [email protected]
Catia Scricciolo Bocconi University, Italy [email protected]
Babak Shahbaba University of California at Irvine, USA [email protected]
Luca Sitzia University of Torino, Italy [email protected]
Dario Spano University of Warwick, UK [email protected]
Mark Steel University of Warwick, UK [email protected]
Erik Sudderth Brown University, USA [email protected]
Matthew Taddy University of Chicago Booth School of Business, USA [email protected]
Mahlet Tadesse Georgetown University, USA [email protected]
Christian Tallberg Karlstad University, Sweden [email protected]
Yee W. Teh University College London, UK [email protected]
Aleksey Tetenov Collegio Carlo Alberto, Italy [email protected]
Surya Tokdar Duke University, USA [email protected]
Takaaki Tokuda Waseda University, Japan [email protected]
Lorenzo Trippa Bocconi University, Italy [email protected]
Aad van der Vaart VU University Amsterdam, Netherlands [email protected]
Stephen G. Walker University of Kent, UK [email protected]
Attilio Wedlin University of Trieste, Italy [email protected]
Sinead Williamson University of Cambridge, UK [email protected]
Fei Xiang University of Kent, UK [email protected]
Hao Wu Free University of Berlin, Germany [email protected]
Oliver Zobay University of Bristol, UK [email protected]
48