Robust Bayesian Inference via Coarsening

Miller, Jeffrey W.; Dunson, David B.

doi:10.1080/01621459.2018.1469995

Cited by 149 publications

(147 citation statements)

References 31 publications

Supporting

Mentioning

141

Contrasting

Order By: Relevance

“…Furthermore, we derive asymptotic settings under which the WABC posterior behaves differently from the posterior by providing upper bounds on concentration as the number of observations goes to ∞ and illustrate the potential effects of model misspecification and the dimension of the observation space. The WABC posterior is a particular case of a coarsened posterior, and our results are complementary to those of Miller and Dunson ().…”

Section: Introductionsupporting

confidence: 86%

“…Therefore, the WABC distribution with a fixed " does not converge to a Dirac mass, in contrast with the standard posterior. As argued in Miller and Dunson (2018), this can have some benefit in case of model misspecification: the WABC posterior is less sensitive to perturbations of the data-generating process than the standard posterior is.…”

Section: Behaviour As N ! 1 For Fixed "mentioning

confidence: 99%

See 1 more Smart Citation

Approximate Bayesian Computation with the Wasserstein Distance

Bernton

Jacob

Gerber

et al. 2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

113

145

View full text Add to dashboard Cite

Summary A growing number of generative statistical models do not permit the numerical evaluation of their likelihood functions. Approximate Bayesian computation has become a popular approach to overcome this issue, in which one simulates synthetic data sets given parameters and compares summaries of these data sets with the corresponding observed values. We propose to avoid the use of summaries and the ensuing loss of information by instead using the Wasserstein distance between the empirical distributions of the observed and synthetic data. This generalizes the well‐known approach of using order statistics within approximate Bayesian computation to arbitrary dimensions. We describe how recently developed approximations of the Wasserstein distance allow the method to scale to realistic data sizes, and we propose a new distance based on the Hilbert space filling curve. We provide a theoretical study of the method proposed, describing consistency as the threshold goes to 0 while the observations are kept fixed, and concentration properties as the number of observations grows. Various extensions to time series data are discussed. The approach is illustrated on various examples, including univariate and multivariate g‐and‐k distributions, a toggle switch model from systems biology, a queuing model and a Lévy‐driven stochastic volatility model.

show abstract

Section: Introductionsupporting

confidence: 86%

Section: Behaviour As N ! 1 For Fixed "mentioning

confidence: 99%

Approximate Bayesian Computation with the Wasserstein Distance

Bernton

Jacob

Gerber

et al. 2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

113

145

View full text Add to dashboard Cite

show abstract

“…We consider the posterior convergence of the Bayesian fractional posterior obtained by raising the likelihood function by a factor η ∈ (0,1] in the Bayes formula

{normalΠ}_{n, η} false(A false) = \frac{true0 \int_{A} \prod_{i = 1}^{n} p_{f} {false(Y_{i} false| X_{i} false)}^{η} normalΠ (d f)}{true0 \int \prod_{i = 1}^{n} p_{f} {false(Y_{i} false| X_{i} false)}^{η} normalΠ (d f)},

where Π denotes the prior probability measure over

L^{2} false({false[0, 1 false]}^{p} false)

: the

{scriptL}^{2}

‐space over [0,1] p . Fractional posteriors have gained renewed attention in Bayesian statistics because of their robustness to model misspecification (Grünwald, ; Miller and Dunson, ). According to Walker and Hjort (), the fractional posterior can be viewed as combining the original likelihood function with a data‐dependent prior that is divided by a portion of the likelihood.…”

Section: Theoretical Resultsmentioning

confidence: 99%

“…[0, 1] p /: the L 2 -space over [0, 1] p . Fractional posteriors have gained renewed attention in Bayesian statistics because of their robustness to model misspecification (Grünwald, 2012;Miller and Dunson, 2018). According to Walker and Hjort (2001), the fractional posterior can be viewed as combining the original likelihood function with a data-dependent prior that is divided by a portion of the likelihood.…”

Section: Theoretical Resultsmentioning

confidence: 99%

Bayesian Regression Tree Ensembles that Adapt to Smoothness and Sparsity

Linero

Yang

2018

Journal of the Royal Statistical Society Series B: Statistical Methodology

103

143

View full text Add to dashboard Cite

Summary Ensembles of decision trees are a useful tool for obtaining flexible estimates of regression functions. Examples of these methods include gradient‐boosted decision trees, random forests and Bayesian classification and regression trees. Two potential shortcomings of tree ensembles are their lack of smoothness and their vulnerability to the curse of dimensionality. We show that these issues can be overcome by instead considering sparsity inducing soft decision trees in which the decisions are treated as probabilistic. We implement this in the context of the Bayesian additive regression trees framework and illustrate its promising performance through testing on benchmark data sets. We provide strong theoretical support for our methodology by showing that the posterior distribution concentrates at the minimax rate (up to a logarithmic factor) for sparse functions and functions with additive structures in the high dimensional regime where the dimensionality of the covariate space is allowed to grow nearly exponentially in the sample size. Our method also adapts to the unknown smoothness and sparsity levels, and can be implemented by making minimal modifications to existing Bayesian additive regression tree algorithms.

show abstract

“…• Bayesian robust inference of mixing measure in finite mixture models has been of interest recently, see for example [Miller and Dunson, 2015]. Whether the idea of minimum Hellinger distance estimator can be adapted to that setting is also an interesting direction to consider in the future.…”

Section: Summaries and Discussionmentioning

confidence: 99%

Robust estimation of mixing measures in finite mixture models

Ho¹,

Nguyen²,

Ritov³

2020

Bernoulli

View full text Add to dashboard Cite

In finite mixture models, apart from underlying mixing measure, true kernel density function of each subpopulation in the data is, in many scenarios, unknown. Perhaps the most popular approach is to choose some kernel functions that we empirically believe our data are generated from and use these kernels to fit our models. Nevertheless, as long as the chosen kernel and the true kernel are different, statistical inference of mixing measure under this setting will be highly unstable. To overcome this challenge, we propose flexible and efficient robust estimators of the mixing measure in these models, which are inspired by the idea of minimum Hellinger distance estimator, model selection criteria, and superefficiency phenomenon. We demonstrate that our estimators consistently recover the true number of components and achieve the optimal convergence rates of parameter estimation under both the well-and mis-specified kernel settings for any fixed bandwidth. These desirable asymptotic properties are illustrated via careful simulation studies with both synthetic and real data. 1

show abstract

Robust Bayesian Inference via Coarsening

Cited by 149 publications

References 31 publications

Approximate Bayesian Computation with the Wasserstein Distance

Approximate Bayesian Computation with the Wasserstein Distance

Bayesian Regression Tree Ensembles that Adapt to Smoothness and Sparsity

Robust estimation of mixing measures in finite mixture models

Contact Info

Product

Resources

About