Sparse Bayesian Binary Logistic Regression Using the Split-and-Augmented Gibbs Sampler

Vono, Maxime; Dobigeon, Nicolas; Chainais, Pierre

doi:10.1109/mlsp.2018.8516963

Cited by 12 publications

(15 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…results of experiments aimed at comparing the proposed methodology with that of current state-of-the-art (optimization and Bayesian) methods for the inverse problems discussed in Section IV. All the results presented in this section have been obtained using MATLAB, on a computer equipped with an Intel Xeon 3.70 GHz processor, with 16.0 GB of RAM, and running Windows 7.Other examples of the proposed approach on machine learning problems can be found in[30],[49].A. Deconvolution with a smooth prior 1) Problem considered: The Gaussian sampling problem introduced in Section IV-B is considered.…”

mentioning

confidence: 99%

Split-and-Augmented Gibbs Sampler—Application to Large-Scale Inference Problems

Vono

Dobigeon

Chainais

2019

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

This paper derives two new optimization-driven Monte Carlo algorithms inspired from variable splitting and data augmentation. In particular, the formulation of one of the proposed approaches is closely related to the alternating direction method of multipliers (ADMM) main steps. The proposed framework enables to derive faster and more efficient sampling schemes than the current state-of-theart methods and can embed the latter. By sampling efficiently the parameter to infer as well as the hyperparameters of the problem, the generated samples can be used to approximate Bayesian estimators of the parameters to infer. Additionally, the proposed approach brings confidence intervals at a low cost contrary to optimization methods. Simulations on two often-studied signal processing problems illustrate the performance of the two proposed samplers. All results are compared to those obtained by recent state-of-the-art optimization and MCMC algorithms used to solve these problems. Index TermsBayesian inference, data augmentation, high-dimensional problems, Markov chain Monte Carlo, variable splitting.

show abstract

mentioning

confidence: 99%

Split-and-Augmented Gibbs Sampler—Application to Large-Scale Inference Problems

Vono

Dobigeon

Chainais

2019

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…convexity, differentiability) of the potential functions f1 and f2, the coupling function can be adaptively chosen. For instance, [20,22] consider ϕρ(x) = (2ρ 2 ) −1 ∥x − z∥ 2 2 for its gradient Lipschitz, differentiability and convexity properties while [21] invokes conjugacy arguments to choose ϕρ. For a detailed discussion on the Gibbs sampling procedure, we refer the reader to [20, Section III.A].…”

Section: Gibbs Samplermentioning

confidence: 99%

“…Similarly to the performance results shown in Table 1 for the first experiment, SGS presents similar reconstruction performances as ADMM or P-MYULA and leads to an improvement in computational time. Some additional illustrations of the proposed approach can be found in [20,22,21] where it is shown that the derived Gibbs sampler is more efficient than state-of-the-art MCMC algorithms. Moreover, it can be distributed over multiple nodes with a well-chosen variable splitting strategy.…”

Section: Image Deblurring In the Wavelet Domainmentioning

confidence: 99%

Efficient Sampling through Variable Splitting-inspired Bayesian Hierarchical Models

Vono

Dobigeon

Chainais

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Markov chain Monte Carlo (MCMC) methods are an important class of computation techniques to solve Bayesian inference problems. Much research has been dedicated to scale these algorithms in highdimensional settings by relying on powerful optimization tools such as gradient information or proximity operators. In a similar vein, this paper proposes a new Bayesian hierarchical model to solve large scale inference problems by taking inspiration from variable splitting methods. Similarly to the latter, the derived Gibbs sampler permits to divide the initial sampling task into simpler ones. As a result, the proposed Bayesian framework can lead to a faster sampling scheme than state-of-the-art methods by embedding them. The strength of the proposed methodology is illustrated on two often-studied image processing problems.

show abstract

“…Our approach was initially presented in Rendell et al (2018). A proposal to use essentially the same framework in a serial context has been independently and contemporaneously published by Vono, Dobigeon, and Chainais (2019), who construct a Gibbs sampler via a "variable splitting" approach. Rather than distributing the computation, the authors focus on the setting where b = 1 to obtain a relaxation of the original simulation problem.…”

Section: Introductionmentioning

confidence: 99%

Global Consensus Monte Carlo

Rendell

Johansen

Lee

et al. 2020

Journal of Computational and Graphical Statistics

View full text Add to dashboard Cite

To conduct Bayesian inference with large datasets, it is often convenient or necessary to distribute the data across multiple machines. We consider a likelihood function expressed as a product of terms, each associated with a subset of the data. Inspired by global variable consensus optimization, we introduce an instrumental hierarchical model associating auxiliary statistical parameters with each term, which are conditionally independent given the top-level parameters. One of these top-level parameters controls the unconditional strength of association between the auxiliary parameters. This model leads to a distributed MCMC algorithm on an extended state space yielding approximations of posterior expectations. A tradeoff between computational tractability and fidelity to the original model can be controlled by changing the association strength in the instrumental model. We further propose the use of an SMC sampler with a sequence of association strengths, allowing both the automatic determination of appropriate strengths and for a bias correction technique to be applied. In contrast to similar distributed Monte Carlo algorithms, this approach requires few distributional assumptions. The performance of the algorithms is illustrated with a number of simulated examples. Supplementary materials for this article are available online.

show abstract

Sparse Bayesian Binary Logistic Regression Using the Split-and-Augmented Gibbs Sampler

Cited by 12 publications

References 30 publications

Split-and-Augmented Gibbs Sampler—Application to Large-Scale Inference Problems

Split-and-Augmented Gibbs Sampler—Application to Large-Scale Inference Problems

Efficient Sampling through Variable Splitting-inspired Bayesian Hierarchical Models

Global Consensus Monte Carlo

Contact Info

Product

Resources

About