On the Well-posedness of Bayesian Inverse Problems

Abstract

The subject of this article is the introduction of a new concept of well-posedness of Bayesian inverse problems. The conventional concept of (Lipschitz, Hellinger) well-posedness in [Stuart 2010, Acta Numerica 19, pp. 451-559] is difficult to verify in practice and may be inappropriate in some contexts. Our concept simply replaces the Lipschitz continuity of the posterior measure in the Hellinger distance by continuity in an appropriate distance between probability measures. Aside from the Hellinger distance, we investigate well-posedness with respect to weak convergence, the total variation distance, the Wasserstein distance, and also the Kullback-Leibler divergence. We demonstrate that the weakening to continuity is tolerable and that the generalisation to other distances is important. The main results of this article are proofs of wellposedness with respect to some of the aforementioned distances for large classes of Bayesian inverse problems. Here, little or no information about the underlying model is necessary; making these results particularly interesting for practitioners using black-box models. We illustrate our findings with numerical examples motivated from machine learning and image processing.

On the local Lipschitz stability of Bayesian inverse problems

“…the observational data and approximations of the forward map has been proven in [7,38] for Gaussian and Besov priors and the Hellinger distance. These results have been generalized to heavy-tailed prior measures in [21,22,40] and a continuous dependence in Hellinger distance was recently shown under substantially relaxed conditions in [29]. In the latter work a continuous dependence on the data was also established concerning the Kullback-Leibler divergence between the resulting posteriors.…”

“…the data y and numerical approximations Φ h of Φ, cf. Remark 1, was already stated for the Bayesian inference with additive Gaussian noise and a Gaussian prior µ by Stuart [38] and recently extended by Dashti and Stuart [7] and Latz [29] to a more general setting. Moreover, in [39, Section 4] we can already find a similar result under slightly different assumptions.…”

“…The relation between the parameters and the observable quantities are given by a measurable forwad map which, in combination with an assumed error distribution, determines the likelihood function for the data given the parameter. The Bayesian approach is quite appealing and, in particular, yields a well-posed inverse problem [7,21,22,29,38,40], i.e., its unique solution-the posterior distribution-depends (locally Lipschitz) continuously on the observational data and behaves also robustly w.r.t. (numerical) approximations of the forward map.…”

“…the Wasserstein distance and show how the general robustness result yields a well-posedness of Bayesian inverse problems in Wasserstein distance. For the latter we use the same basic assumptions stated in [7,29,38] for the well-posedness in Hellinger distance.…”

On uniform continuity of posterior distributions

“…While the pioneering paper [Z], essentially inspired by foundational questions, dealt with the qualitative definition of continuity for conditional distributions, more recent studies highlight the relevance of modulus of continuity estimates. We refer, for instance, to the well-posedness theory developed in [S] and to different results found in [DS,CDRS,ILS,L1]. Our contribution moves in the same direction.…”