Consistency of Bayes Estimates for Nonparametric Regression: Normal Theory

Diaconis, Persi; Freedman, David A.

doi:10.2307/3318659

Cited by 28 publications

(17 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When unknown parameters are estimated using priors and sampled data, it is important to observe that the convergence of the Bayesian method may fail if the underlying probability mechanism allows an infinite number of possible outcomes (e.g., estimation of an unknown probability on N, the set of all natural numbers) [17]. In fact, in these infinite-dimensional situations, this lack of convergence (commonly referred to as inconsistency) is the rule rather than the exception [18]. As emphasized in [17], as more data comes in, some Bayesian statisticians will become more and more convinced of the wrong answer.…”

Section: Comparisons With Other Uq Methodsmentioning

confidence: 99%

Optimal Uncertainty Quantification

Scovel²,

et al. 2013

View full text Add to dashboard Cite

We propose a rigorous framework for Uncertainty Quantification (UQ) in which the UQ objectives and the assumptions/information set are brought to the forefront. This framework, which we call Optimal Uncertainty Quantification (OUQ), is based on the observation that, given a set of assumptions and information about the problem, there exist optimal bounds on uncertainties: these are obtained as values of well-defined optimization problems corresponding to extremizing probabilities of failure, or of deviations, subject to the constraints imposed by the scenarios compatible with the assumptions and information. In particular, this framework does not implicitly impose inappropriate assumptions, nor does it repudiate relevant information.Although OUQ optimization problems are extremely large, we show that under general conditions they have finite-dimensional reductions. As an application, we develop Optimal Concentration Inequalities (OCI) of Hoeffding and McDiarmid type. Surprisingly, these results show that uncertainties in input parameters, which propagate to output uncertainties in the classical sensitivity analysis paradigm, may fail to do so if the transfer functions (or probability distributions) are imperfectly known. We show how, for hierarchical structures, this phenomenon may lead to the non-propagation of uncertainties or information across scales.In addition, a general algorithmic framework is developed for OUQ and is tested on the Caltech surrogate model for hypervelocity impact and on the seismic safety assessment of truss structures, suggesting the feasibility of the framework for important complex systems.The introduction of this paper provides both an overview of the paper and a self-contained mini-tutorial about basic concepts and issues of UQ.

show abstract

Section: Comparisons With Other Uq Methodsmentioning

confidence: 99%

Optimal Uncertainty Quantification

Scovel²,

et al. 2013

View full text Add to dashboard Cite

show abstract

“…However, there are also analytical reasons to be careful about the application of Bayesian methods [88,76,43]. It is, in fact, now well understood that Bayesian methods may fail to converge or may converge towards the wrong solution if the underlying probability mechanism allows an infinite number of possible outcomes [35] and that, in these non-finite-probability-space situations, this lack of convergence (commonly referred to as Bayesian inconsistency) is the rule rather than the exception [36]. There is now a wide literature of positive [19,30,38,67,69,96,92] and negative results [12,35,48,47,61,71] on the consistency properties of Bayesian inference in parametric and non-parametric settings, and an emerging understanding of the fine topological and geometrical properties that determine (in)consistency.…”

Section: Bayesian Inconsistency and Model Misspecificationmentioning

confidence: 99%

Brittleness of Bayesian inference under finite information in a continuous world

Owhadi¹,

Scovel²,

Sullivan³

2015

Electron. J. Statist.

View full text Add to dashboard Cite

We derive, in the classical framework of Bayesian sensitivity analysis, optimal lower and upper bounds on posterior values obtained from Bayesian models that exactly capture an arbitrarily large number of finite-dimensional marginals of the datagenerating distribution and/or that are as close as desired to the data-generating distribution in the Prokhorov or total variation metrics; these bounds show that such models may still make the largest possible prediction error after conditioning on an arbitrarily large number of sample data measured at finite precision. These results are obtained through the development of a reduction calculus for optimization problems over measures on spaces of measures. We use this calculus to investigate the mechanisms that generate brittleness/robustness and, in particular, we observe that learning and robustness are antagonistic properties. It is now well understood that the numerical resolution of PDEs requires the satisfaction of specific stability conditions. Is there a missing stability condition for using Bayesian inference in a continuous world under finite information?Contents 2010 Mathematics Subject Classification: 62A01, 62E20, 62F12, 62F15, 62G20, 62G35.

show abstract

“…Although it is known from the results of Diaconis and Freedman that the Bayesian method may fail to converge or may converge towards the wrong solution (i.e., be inconsistent) if the underlying probability mechanism allows an infinite number of possible outcomes [14] and that in these non-finite-probability-space situations, this lack of convergence (commonly referred to as Bayesian inconsistency) is the rule rather than the exception [15], it is also known, from the Bernstein-Von Mises Theorem [7,42] (see also LeCam [28]), that consistency (convergence upon observation of sample data) does indeed hold, under some regularity conditions, if the data-generating distribution of the sample data belongs to the finite dimensional family of distributions parameterized by the model. Furthermore, although it is also known that this convergence may fail under model misspecification [43,21,32,1,2,26,29,22] (i.e.…”

Section: Introductionmentioning

confidence: 99%

Brittleness of Bayesian inference and new Selberg formulas

Owhadi

Scovel

2016

Communications in Mathematical Sciences

View full text Add to dashboard Cite

The incorporation of priors [30] in the Optimal Uncertainty Quantification (OUQ) framework [31] reveals brittleness in Bayesian inference; a model may share an arbitrarily large number of finite-dimensional marginals with, or be arbitrarily close (in Prokhorov or total variation metrics) to, the data-generating distribution and still make the largest possible prediction error after conditioning on an arbitrarily large number of samples. The initial purpose of this paper is to unwrap this brittleness mechanism by providing (i) a quantitative version of the Brittleness Theorem of [30] and (ii) a detailed and comprehensive analysis of its application to the revealing example of estimating the mean of a random variable on the unit interval [0, 1] using priors that exactly capture the distribution of an arbitrarily large number of Hausdorff moments.However, in doing so, we discovered that the free parameter associated with Markov and Kreȋn's canonical representations of truncated Hausdorff moments generates reproducing kernel identities corresponding to reproducing kernel Hilbert spaces of polynomials. Furthermore, these reproducing identities lead to biorthogonal systems of Selberg integral formulas.This process of discovery appears to be generic: whereas Karlin and Shapley used Selberg's integral formula to first compute the volume of the Hausdorff moment space (the polytope defined by the first n moments of a probability measure on the interval [0, 1]), we observe that the computation of that volume along with higher order moments of the uniform measure on the moment space, using different finite-dimensional representations of subsets of the infinite-dimensional set of probability measures on [0, 1] representing the first n moments, leads to families of equalities corresponding to classical and new Selberg identities.

show abstract

Consistency of Bayes Estimates for Nonparametric Regression: Normal Theory

Cited by 28 publications

References 37 publications

Optimal Uncertainty Quantification

Optimal Uncertainty Quantification

Brittleness of Bayesian inference under finite information in a continuous world

Brittleness of Bayesian inference and new Selberg formulas

Contact Info

Product

Resources

About