Gibbs posterior inference on multivariate quantiles

Bhattacharya, Indrabati; Martin, Ryan

doi:10.1016/j.jspi.2021.10.003

Cited by 13 publications

(15 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BvM for median and quantiles under classical and Gibbs posterior [4], [5]. Interestingly, [46] show that Bayesian neural networks show inconsistency similar to that discussed in [16], applying variational Bayes leads to BNN becoming consistent; it would be interesting to study whether it is possible to achieve asymptotic efficiency.…”

Section: Discussion and Open Questionsmentioning

confidence: 99%

Bernstein - von Mises theorem and misspecified models: a review

Bochkina¹

2022

Preprint

View full text Add to dashboard Cite

IntroductionConsider a family of probability models P (Y | θ) indexed by parameter θ ∈ Θ for observations y, and a prior distribution π on the parameter space Θ. In a classical Bayesian approach, the posterior distributionDenote the true distribution of observations P 0 , and we consider the casethe model is misspecified. Such case arises in many applications, particularly in complex models where the numerical evaluation of posterior distribution under the ideal probability model takes a long time to run, leading to increased use of approximate models with faster computing time. A typical example is approximating complex dependence structure by pairwise dependence only [2], [40].For well-specified regular models, the classical Bernstein -von Mises theorem states that for n independent identically distributed (iid) observations, for large n, the posterior distribution behaves approximately as a normal distribution centered on the true value of the parameter with a random shift, with both the posterior variance and the variance of the random shift being asymptotically equal to the inverse Fisher information, thus making the Bayesian inference asymptotically consistent and efficient.This was extended to locally asymptotically normal (LAN) models for Θ ∈ R p with fixed p [39], for LAN models with growing p, etc.A version of Bernstein -von Mises theorem is also available for nonregular models such as locally asymptotically exponential models ([20], [9]), models with parameter on the boundary of the parameter space [7] which hold under misspecified models. Here, however, we focus on "regular" models where estimators are asymptotically Gaussian.However, under model misspecification, Bayesian model is no longer optimal, as the posterior variance does not match the minimal lower bound on the variance of unbiased estimators [22] and [29]).Therefore, the standard way of constructing a posterior distribution following the Bayes theorem may not be appropriate for particular purposes, e.g. inference or prediction [27]. Different ways to construct a distribution of θ given Y that produces inference appropriate for the purpose of the analysis have been proposed. A natural aim for such a method is to behave like a standard Bayesian method when the

show abstract

Section: Discussion and Open Questionsmentioning

confidence: 99%

Bernstein - von Mises theorem and misspecified models: a review

Bochkina¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Beyond consistency and rates, there are cases in which the Gibbs posterior enjoys a version of the celebrated Bernstein-von Mises theorem, i.e., that the Gibbs posterior takes on a Gaussian shape asymptotically. This was demonstrated for a special case in Bhattacharya and Martin (2022) but their results are generalized below.…”

Section: Distributional Approximationsmentioning

confidence: 74%

“…In Bhattacharya and Martin (2022), the effect of η on the asymptotic Gibbs posterior mean was overlooked-they stated the posterior mean was θn instead of η θn + (1 − η)θ as in (17). This small effect went unnoticed because the learning rate suggested in the former case ends up being larger than in the latter, hence more conservative Gibbs posterior credible regions.…”

mentioning

confidence: 99%

Direct Gibbs posterior inference on risk minimizers: construction, concentration, and calibration

Martin¹,

Syring²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Real-world problems, often couched as machine learning applications, involve quantities of interest that have real-world meaning, independent of any statistical model. To avoid potential model misspecification bias or over-complicating the problem formulation, a direct, model-free approach is desired. The traditional Bayesian framework relies on a model for the data-generating process so, apparently, the desired direct, model-free, posterior-probabilistic inference is out of reach. Fortunately, likelihood functions are not the only means of linking data and quantities of interest. Loss functions provide an alternative link, where the quantity of interest is defined, or at least could be defined, as a minimizer of the corresponding risk, or expected loss. In this case, one can obtain what is commonly referred to as a Gibbs posterior distribution by using the empirical risk function directly. This manuscript explores the Gibbs posterior construction, its asymptotic concentration properties, and the frequentist calibration of its credible regions. By being free from the constraints of model specification, Gibbs posteriors create new opportunities for probabilistic inference in modern statistical learning problems.

show abstract

“…In many cases, it is more natural to formulate the inference problem with a loss function rather than a statistical model. These are often referred to as Gibbs posterior distributions; see Syring and Martin (2017, 2019, 2020b, Bhattacharya and Martin (2022), Wang and Martin (2020), and Section 6 below.…”

Section: Generalized Bayesmentioning

confidence: 99%

A Comparison of Learning Rate Selection Methods in Generalized Bayesian Inference

Martin

2023

Bayesian Anal.

Self Cite

View full text Add to dashboard Cite

Generalized Bayes posterior distributions are formed by putting a fractional power on the likelihood before combining with the prior via Bayes's formula. This fractional power, which is often viewed as a remedy for potential model misspecification bias, is called the learning rate, and a number of data-driven learning rate selection methods have been proposed in the recent literature. Each of these proposals has a different focus, a different target they aim to achieve, which makes them difficult to compare. In this paper, we provide a direct head-to-head empirical comparison of these learning rate selection methods in various misspecified model scenarios, in terms of several relevant metrics, in particular, coverage probability of the generalized Bayes credible regions. In some examples all the methods perform well, while in others the misspecification is too severe to be overcome, but we find that the so-called generalized posterior calibration algorithm tends to outperform the others in terms of credible region coverage probability.

show abstract

Gibbs posterior inference on multivariate quantiles

Cited by 13 publications

References 32 publications

Bernstein - von Mises theorem and misspecified models: a review

Bernstein - von Mises theorem and misspecified models: a review

Direct Gibbs posterior inference on risk minimizers: construction, concentration, and calibration

A Comparison of Learning Rate Selection Methods in Generalized Bayesian Inference

Contact Info

Product

Resources

About