On Choosing Mixture Components via Non-Local Priors

Fuquene, Jairo; Steel, Mark F. J.; Rossell, David

doi:10.1111/rssb.12333

Cited by 15 publications

(11 citation statements)

References 57 publications

(136 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non-local priors possess appealing properties for Bayesian model selection. They discard spurious parameters faster as the sample size n grows, but preserve exponential rates to detect important coefficients (Johnson and Rossell, 2010;Fúquene et al, 2018) and can lead to improved parameter estimation shrinkage (Rossell and Telesca, 2017). To illustrate the motivation for NLPs in our setting consider Figure 1.…”

Section: Non-local Spike-and-slab Priormentioning

confidence: 99%

Heterogeneous Large Datasets Integration Using Bayesian Factor Regression

2022

Self Cite

View full text Add to dashboard Cite

Two key challenges in modern statistical applications are the large amount of information recorded per individual, and that such data are often not collected all at once but in batches. These batch effects can be complex, causing distortions in both mean and variance. We propose a novel sparse latent factor regression model to integrate such heterogeneous data. The model provides a tool for data exploration via dimensionality reduction and sparse low-rank covariance estimation while correcting for a range of batch effects. We study the use of several sparse priors (local and non-local) to learn the dimension of the latent factors. We provide a flexible methodology for sparse factor regression which is not limited to data with batch effects. Our model is fitted in a deterministic fashion by means of an EM algorithm for which we derive closed-form updates, contributing a novel scalable algorithm for non-local priors of interest beyond the immediate scope of this paper. We present several examples, with a focus on bioinformatics applications. Our results show an increase in the accuracy of the dimensionality reduction, with non-local priors substantially improving the reconstruction of factor cardinality. The results of our analyses illustrate how failing to properly account for batch effects can result in unreliable inference. Our model provides a novel approach to latent factor regression that balances sparsity with sensitivity in scenarios both with and without batch effects and is highly computationally efficient.

show abstract

Section: Non-local Spike-and-slab Priormentioning

confidence: 99%

Heterogeneous Large Datasets Integration Using Bayesian Factor Regression

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…The relative entropy function D has values equal to D (μ || ν) = dμ log (dμ/dν), the entropy of a probability measure μ relative to a measure ν that dominates μ (e.g., Maas, 2017). Complementary statistical applications of relative entropy include the prevention of overfitting models (Fúquene et al, 2016;Gelman et al, 2017), the idealization of Cromwell's rule for revising priors (Bickel, 2018), and the automatic construction of unsharpened priors (Section 8, Example 7).…”

Section: Adjusting Priors For the Simplicity Of Data Pdfsmentioning

confidence: 99%

An Explanatory Rationale for Priors Sharpened Into Occam’s Razors

Bickel¹

2020

Bayesian Anal.

View full text Add to dashboard Cite

In Bayesian statistics, if the distribution of the data is unknown, then each plausible distribution of the data is indexed by a parameter value, and the prior distribution of the parameter is specified. To the extent that more complicated data distributions tend to require more coincidences for their construction than simpler data distributions, default prior distributions should be transformed to assign additional prior probability or probability density to the parameter values that refer to simpler data distributions. The proposed transformation of the prior distribution relies on the entropy of each data distribution as the relevant measure of complexity. The transformation is derived from a few first principles and extended to stochastic processes.

show abstract

“…This assumption, although convenient for mathematical tractability, is often an oversimplification and might produce misleading results in producing too many clusters. This motivated Petralia et al (2012), Xu et al (2016), Fúquene et al (2019), Quinlan et al (2020), Bianchini et al (2020), and Xie and Xu (2019) to explicitly define prior models with repulsion between the locations, thereby obtaining well separated components.…”

Section: Previous Work On Repulsive Mixture Modelsmentioning

confidence: 99%

“…In Petralia et al (2012), Fúquene et al (2019), andQuinlan et al (2020), m is finite and fixed, but, as mentioned before, this cannot guarantee posterior consistency of the number of components. However, Xu et al (2016), Bianchini et al (2020), and Xie and Xu (2019) assumed m to be finite and random.…”

Section: Previous Work On Repulsive Mixture Modelsmentioning

confidence: 99%

MCMC computations for Bayesian mixture models using repulsive point processes

Beraha¹,

Argiento²,

Møller³

et al. 2020

Preprint

View full text Add to dashboard Cite

Repulsive mixture models have recently gained popularity for Bayesian cluster detection. Compared to more traditional mixture models, there is empirical evidence suggesting that repulsive mixture models produce a smaller number of well separated clusters. The most commonly used methods for posterior inference either require to fix a priori the number of components or are based on reversible jump MCMC computation. We present a general framework for mixture models, when the prior of the 'cluster centres' is a finite point process depending on a hyperparameter -not only a Poisson or determinantal point process (DPP) as previously considered in the literature but also a repulsive point process specified by a density which depends on an intractable normalizing constant. By investigating the posterior characterization of this class of mixture models, we derive a MCMC algorithm which avoids the well-known difficulties associated to reversible jump MCMC computation. In particular, when the point process density involves an intractable normalizing constant, we use an ancillary variable method which eliminate the problem of having a ratio of normalizing constants in the Hastings ratio when making posterior simulations for full conditional of the hyperparameter. The ancillary variable method relies on a perfect simulation algorithm, and we demonstrate this is fast because the number of components is typically small. In several simulation studies and an application on sociological data, we illustrate the advantage of our new methodology over existing methods, and we compare the use of a DPP or a repulsive Gibbs point process prior model.

show abstract

On Choosing Mixture Components via Non-Local Priors

Cited by 15 publications

References 57 publications

Heterogeneous Large Datasets Integration Using Bayesian Factor Regression

Heterogeneous Large Datasets Integration Using Bayesian Factor Regression

An Explanatory Rationale for Priors Sharpened Into Occam’s Razors

MCMC computations for Bayesian mixture models using repulsive point processes

Contact Info

Product

Resources

About