High-dimensional Gaussian sampling: a review and a unifying approach based on a stochastic proximal point algorithm

Abstract

Efficient sampling from a high-dimensional Gaussian distribution is an old but high-stake issue. In past years, multiple methods have been proposed from different communities to tackle this difficult sampling task ranging from iterative numerical linear algebra to Markov chain Monte Carlo (MCMC) approaches. Surprisingly, no complete review and comparison of these methods have been conducted. This paper aims at reviewing all these approaches by pointing out their differences, close relations, benefits and limitations. In addition to this state of the art, this paper proposes a unifying Gaussian simulation framework by deriving a stochastic counterpart of the celebrated proximal point algorithm in optimization. This framework offers a novel and unifying revisit of most of the existing MCMC approaches while extending them. Guidelines to choose the appropriate Gaussian simulation method for a given sampling problem are proposed and illustrated with numerical examples.

DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs

“…Under H1, note that Q is invertible and therefore this conditional Gaussian distribution is well-defined. Since sampling from high-dimensional Gaussian distributions can be performed efficiently (Vono et al, 2020), this Gibbs sampling scheme is interesting as long as sampling from (4) is cheap. proposed the use of a rejection sampling step requiring to set…”