Brittleness of Bayesian inference and new Selberg formulas

Abstract

The incorporation of priors [30] in the Optimal Uncertainty Quantification (OUQ) framework [31] reveals brittleness in Bayesian inference; a model may share an arbitrarily large number of finite-dimensional marginals with, or be arbitrarily close (in Prokhorov or total variation metrics) to, the data-generating distribution and still make the largest possible prediction error after conditioning on an arbitrarily large number of samples. The initial purpose of this paper is to unwrap this brittleness mechanism by providing (i) a quantitative version of the Brittleness Theorem of [30] and (ii) a detailed and comprehensive analysis of its application to the revealing example of estimating the mean of a random variable on the unit interval [0, 1] using priors that exactly capture the distribution of an arbitrarily large number of Hausdorff moments.However, in doing so, we discovered that the free parameter associated with Markov and Kreȋn's canonical representations of truncated Hausdorff moments generates reproducing kernel identities corresponding to reproducing kernel Hilbert spaces of polynomials. Furthermore, these reproducing identities lead to biorthogonal systems of Selberg integral formulas.This process of discovery appears to be generic: whereas Karlin and Shapley used Selberg's integral formula to first compute the volume of the Hausdorff moment space (the polytope defined by the first n moments of a probability measure on the interval [0, 1]), we observe that the computation of that volume along with higher order moments of the uniform measure on the moment space, using different finite-dimensional representations of subsets of the infinite-dimensional set of probability measures on [0, 1] representing the first n moments, leads to families of equalities corresponding to classical and new Selberg identities.

On the Brittleness of Bayesian Inference

“…When applied to Example 3 with the set Π := Ψ −1 Q of priors generated instead by the uniform prior Q restricted to the truncated moment space, Theorem 3.3 of [46] establishes that, although the prior value satisfies U(Π) = 1 2 , the posterior value satisfies…”

“…To quantify "large enough" and "small enough" and to remove the "nonatomic" requirement above, Theorem 3.1 of [46] provides a quantitative version of Theorem 2 in which the conditions of the theorem are only required to hold approximately. When applied to Example 3 with the set Π := Ψ −1 Q of priors generated instead by the uniform prior Q restricted to the truncated moment space, Theorem 3.3 of [46] establishes that, although the prior value satisfies U(Π) = 1 2 , the posterior value satisfies…”

“…To study this problem, we introduce a representation space Q (e.g., prototypically, R k ) and a mapping Ψ : A → Q from the subset A ⊆ M(X ) into Q, which can be 4 All results of this article and those in [46,47,48] require some mild technical measure-theoretic and topological assumptions. For example, here it is sufficient if P(Θ) is a Borel subset of a Polish space (a separable completely metrizable space).…”

“…This paper summarizes recent results [46,47] that give conditions under which Bayesian inference appears to be nonrobust in the most extreme fashion, in the sense that arbitrarily small changes of the prior and model class lead to arbitrarily large changes of the posterior value of a quantity of interest. We call this extreme nonrobustness "brittleness," and it can be visualized as the smooth dependence of the value of the quantity of interest on the prior breaking into a fine patchwork, in which nearby priors are associated to diametrically opposed posterior values.…”

Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients

“…5] of the linearity of the PDE and the quadratic nature of the loss function. For non linear PDEs or non quadratic loss functions, although optimal priors (which may not be Gaussian) could in principle be numerically approximated, such approximations could be severely impacted by stability issues as discussed in [67,63,68,64]. , Ω = (0, 1) 2 and T h is a square grid of mesh size h = (1 + 2 q ) −1 with r = 6 and 64 × 64 interior nodes, a is piecewise constant on each square of T h and given by a(x) = Π r k=1 1+0.5 cos(2 k π( Figure 2.…”