Proximal nested sampling for high-dimensional Bayesian model selection

Abstract

Bayesian model selection provides a powerful framework for objectively comparing models directly from observed data, without reference to ground truth data. However, Bayesian model selection requires the computation of the marginal likelihood (model evidence), which is computationally challenging, prohibiting its use in many high-dimensional Bayesian inverse problems. With Bayesian imaging applications in mind, in this work we present the proximal nested sampling methodology to objectively compare alternative Bayesian imaging models for applications that use images to inform decisions under uncertainty. The methodology is based on nested sampling, a Monte Carlo approach specialised for model comparison, and exploits proximal Markov chain Monte Carlo techniques to scale efficiently to large problems and to tackle models that are log-concave and not necessarily smooth (e.g., involving $$\ell _1$$ ℓ 1 or total-variation priors). The proposed approach can be applied computationally to problems of dimension $${\mathcal {O}}(10^6)$$ O ( 10 6 ) and beyond, making it suitable for high-dimensional inverse imaging problems. It is validated on large Gaussian models, for which the likelihood is available analytically, and subsequently illustrated on a range of imaging problems where it is used to analyse different choices of dictionary and measurement model.

Proximal Nested Sampling with Data-Driven Priors for Physical Scientists

“…Proximal nested sampling addresses this challenge for the case of log-convex likelihoods, which are widespread in computational imaging problems. In this section we review the proximal nested sampling framework [12] in a pedagogical manner, sacrificing some mathematical rigor in an attempt to improve readability and accessibility.…”

“…The proximal nested sampling framework follows by taking the constrained sampling formulation of Equation ( 6), adopting Langevin MCMC sampling of Equation ( 8), and applying Moreau-Yosida regularisation of Equation ( 4) to the convex constraint χ B τ (x) to yield a differentiable target. This strategy yields (see [12]) the update equation:…”

“…With the current formulation we are not guaranteed to recover samples from the prior subject to the hard likelihood constraint due to the approximation introduced in the Moreu-Yosida regularisation and due to the approximation in discretising the underlying Langevin stochastic differential equation. We therefore introduce a Metropolis-Hastings correction step to eliminate the bias introduced by these approximations and ensure convergence to the required target density (see [12] for further details).…”

“…While a number of highly successful approaches to computing the model evidence have been developed, such as nested sampling (e.g., [2][3][4][5][6][7][8]) and the learned harmonic mean estimator [9][10][11], previous approaches do not scale to the very high-dimensional settings of computational imaging, which is our driving motivation. The proximal nested sampling framework was introduced recently by a number of authors of the current article in order to open up Bayesian model selection for highdimensional imaging problems [12]. Proximal nested sampling is suitable for models for which the likelihood is log-convex, which are ubiquitous in the imaging sciences.…”

Posterior sampling for inverse imaging problems on the sphere in seismology and cosmology

“…tributions. Proximal mappings have also recently been used to scale nested sampling methods to high-dimensional imaging problems (Cai, McEwen & Pereyra 2022).…”