Wasserstein stability estimates for covariance-preconditioned Fokker–Planck equations

Carrillo, José A.; Vaes, Urbain

doi:10.1088/1361-6544/abbe62

Cited by 16 publications

(18 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is in contrast with the sampling and inversion methods for inverse problems that are based on the ensemble Kalman filter, essentially because these methods are affine-invariant [21]: they behave similarly across the class of problems that differ only by an affine transformation. Ensemble Kalman methods can be viewed, at least in the case of a linear forward model, as coupled gradient descents dynamics or overdamped Langevin diffusions preconditioned by the covariance of the ensemble, providing good stability and convergence properties [20,10].…”

Section: Resultsmentioning

confidence: 99%

“…EKI, EKS and ALDI, enjoy the property of being affine invariant in the sense of [22]; see also [43]. This property was studied carefully in [21] for EKS and ALDI, after it had been observed that, in the simple case of a linear forward model, the convergence rate of EKI and EKS was independent of the Hessian of the regularized least-squares functional Φ R [20,10]. As the terminology indicates, affine invariant methods are insensitive to affine transformations of the regularized least-squares functional Φ R , which makes them particularly well-suited in cases where the Φ R exhibits strong anisotropy at its minimizer.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Derivative-Free Bayesian Inversion Using Multiscale Dynamics

Pavliotis¹,

Stuart²,

Vaes³

2022

SIAM J. Appl. Dyn. Syst.

View full text Add to dashboard Cite

Inverse problems are ubiquitous because they formalize the integration of data with mathematical models. In many scientific applications the forward model is expensive to evaluate, and adjoint computations are difficult to employ; in this setting derivative-free methods which involve a small number of forward model evaluations are an attractive proposition. Ensemble Kalman based interacting particle systems (and variants such as consensus based and unscented Kalman approaches) have proven empirically successful in this context, but suffer from the fact that they cannot be systematically refined to return the true solution, except in the setting of linear forward models [20]. In this paper, we propose a new derivative-free approach to Bayesian inversion, which may be employed for posterior sampling or for maximum a posteriori (MAP) estimation, and may be systematically refined. The method relies on a fast/slow system of stochastic differential equations (SDEs) for the local approximation of the gradient of the log-likelihood appearing in a Langevin diffusion. Furthermore the method may be preconditioned by use of information from ensemble Kalman based methods (and variants), providing a methodology which leverages the documented advantages of those methods, whilst also being provably refineable. We define the methodology, highlighting its flexibility and many variants, provide a theoretical analysis of the proposed approach, and demonstrate its efficacy by means of numerical experiments.

show abstract

Section: Resultsmentioning

confidence: 99%

mentioning

confidence: 99%

Derivative-Free Bayesian Inversion Using Multiscale Dynamics

Pavliotis¹,

Stuart²,

Vaes³

2022

SIAM J. Appl. Dyn. Syst.

View full text Add to dashboard Cite

show abstract

“…for any x, y ∈ R d ; see similar computations in [26,13]. The first identity can be checked directly, and the second identity follows e.g.…”

Section: Propagation Of Gaussiansmentioning

confidence: 87%

“…An alternative Kalman methodology (ensemble Kalman inversion -the EKI) for the optimization approach to the inverse problem, which involves iteration to infinity, was introduced and studied in [34,33] in discrete time and in [58,59] in continuous time; the idea of using ensemble methods for optimization rather than sampling was anticipated in [55]. The ensemble based optimization approach was generalized to approximate sampling of the Bayesian posterior solution to the inverse problem in [26] (the ensemble Kalman sampler -the EKS), and studied further in [13,27,49].…”

Section: Literature Reviewmentioning

confidence: 99%

Consensus Based Sampling

Carrillo¹,

Hoffmann²,

Stuart³

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a novel method for sampling and optimization tasks based on a stochastic interacting particle system. We explain how this method can be used for the following two goals: (i) generating approximate samples from a given target distribution; (ii) optimizing a given objective function. The approach is derivative-free and affine invariant, and is therefore well-suited for solving inverse problems defined by complex forward models: (i) allows generation of samples from the Bayesian posterior and (ii) allows determination of the maximum a posteriori estimator. We investigate the properties of the proposed family of methods in terms of various parameter choices, both analytically and by means of numerical simulations. The analysis and numerical simulation establish that the method has potential for general purpose optimization tasks over Euclidean space; contraction properties of the algorithm are established under suitable conditions, and computational experiments demonstrate wide basins of attraction for various specific problems. The analysis and experiments also demonstrate the potential for the sampling methodology in regimes in which the target distribution is unimodal and close to Gaussian; indeed we prove that the method recovers a Laplace approximation to the measure in certain parametric regimes and provide numerical evidence that this Laplace approximation attracts a large set of initial conditions in a number of examples.

show abstract

“…Gradient-free methods, which rely only on evaluations of the loss function, are therefore an attractive alternative to SGD methods in these settings. In this paper, we consider two particular classes of methods belonging to this category which received a lot of attention lately: the Consensus-Based Optimization (CBO) methods [47,11,14,12], reviewed recently in [53], and methods based on the Ensemble Kalman Filter (EnKF) [17,21,34,51,25,26,13], which are mainly employed in the context of inverse problems but have also proved useful for machine learning tasks [41]. It is shown in [20] that gradient-free ensemble Kalman methods perform better than their gradient-based counterparts in noisy likelihood landscapes.…”

Section: Introductionmentioning

confidence: 99%

Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints

Carrillo¹,

Totzeck²,

Vaes³

2021

Preprint

View full text Add to dashboard Cite

We introduce a practical method for incorporating equality and inequality constraints in global optimization methods based on stochastic interacting particle systems, specifically consensusbased optimization (CBO) and ensemble Kalman inversion (EKI). Unlike other approaches in the literature, the method we propose does not constrain the dynamics to the feasible region of the state space at all times; the particles evolve in the full space, but are attracted towards the feasible set by means of a penalization term added to the objective function and, in the case of CBO, an additional relaxation drift. We study the properties of the method through the associated mean-field Fokker-Planck equation and demonstrate its performance in numerical experiments on several test problems.

show abstract

Wasserstein stability estimates for covariance-preconditioned Fokker–Planck equations

Cited by 16 publications

References 24 publications

Derivative-Free Bayesian Inversion Using Multiscale Dynamics

Derivative-Free Bayesian Inversion Using Multiscale Dynamics

Consensus Based Sampling

Consensus-based Optimization and Ensemble Kalman Inversion for Global Optimization Problems with Constraints

Contact Info

Product

Resources

About