Distances between nested densities and a measure of the impact of the prior in Bayesian statistics

Abstract

In this paper we propose tight upper and lower bounds for the Wasserstein distance between any two univariate continuous distributions with probability densities p 1 and p 2 having nested supports. These explicit bounds are expressed in terms of the derivative of the likelihood ratio p 1 /p 2 as well as the Stein kernel τ 1 of p 1 . The method of proof relies on a new variant of Stein's method which manipulates Stein operators.We give several applications of these bounds. Our main application is in Bayesian statistics : we derive explicit data-driven bounds on the Wasserstein distance between the posterior distribution based on a given prior and the no-prior posterior based uniquely on the sampling distribution. This is the first finite sample result confirming the well-known fact that with well-identified parameters and large sample sizes, reasonable choices of prior distributions will have only minor effects on posterior inferences if the data are benign.

Quantification of the impact of priors in Bayesian statistics via Stein’s Method

“…Our proof relies on (2) and (3). The function π 0 from Ley et al [3] coincides in our case with κ 2 (x) κ 1 (x) ρ(θ), yielding our conditions on ρ(θ) and the upper bound…”

“…One can readily see that this function is always positive and vanishes at the boundaries of the support. For more information about Stein kernels, we refer the reader to Ley et al [3]; for the present paper no further knowledge than the definition (1) is necessary.…”

First-order covariance inequalities via Stein’s method

“…The backbone of the present paper consists in the approach from [50,53,62]. Before introducing these results, we fix the notations.…”

“…We clarify these assumptions in Section 2.3. For more details on Stein class and operators, we refer to [53] for the construction in an abstract setting, [50] for the construction in the continuous setting (i.e. = 0) and [33] for the construction in the discrete setting (i.e.…”

Stein’s method and the distribution of the product of zero mean correlated normal random variables

“…Over the years, Stein characterisations have been obtained for many classical probability distributions (for an overview see Gaunt, Mijoule and Swan [12] and Ley, Reinert and Swan [17]), and also recently for more exotic distributions, such as linear combinations of gamma random variables (Arras et al [4]) and products of independent normal, beta and gamma random variables (Gaunt [9,10]), for which it is difficult to write down a formula for the PDF of the distribution. Stein characterisations of probability distributions are most commonly used as part of Stein's method to derive distributional approximations, with powerful applications in random graph and network theory (Franceschetti and Meester [8]), convergence rates in classical asymptotic results in statistics (Anastasiou and Reinert [1], Gaunt, Pickett and Reinert [14]), Bayesian statistics (Ley, Reinert and Swan [18]) and statistical learning and inference (Gorham et al [15]); see the survey Ross [24] for a list of further application areas. However, recently Gaunt [10] and Gaunt, Mijoule and Swan [12] have found a novel application for Stein characterisations, in which they are used to establish formulas for PDFs of distributions that are too difficult to obtain via other methods.…”