Mean field variational Bayes for continuous sparse signal shrinkage: Pitfalls and remedies

Neville, Sarah Elizabeth; Ormerod, John T.; Wand, M. P.

doi:10.1214/14-ejs910

Cited by 43 publications

(42 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notable exceptions are Armagan, Dunson & Clyde (2011) and Neville, Ormerod & Wand (2013). Neville, Ormerod & Wand (2013) develop algorithms to improve the performance of VB methods with regard to several continuous shrinkage priors. In this paper we do not compare the performance of other priors to the one we choose, but show that VB estimators in this simple Bayesian model can exhibit some desirable properties.…”

Section: Introductionmentioning

confidence: 99%

On Variational Bayes Estimation and Variational Information Criteria for Linear Regression Models

You

Ormerod

Müller

2014

Aust. N. Z. J. Stat.

Self Cite

View full text Add to dashboard Cite

Summary Variational Bayes (VB) estimation is a fast alternative to Markov Chain Monte Carlo for performing approximate Baesian inference. This procedure can be an efficient and effective means of analyzing large datasets. However, VB estimation is often criticised, typically on empirical grounds, for being unable to produce valid statistical inferences. In this article we refute this criticism for one of the simplest models where Bayesian inference is not analytically tractable, that is, the Bayesian linear model (for a particular choice of priors). We prove that under mild regularity conditions, VB based estimators enjoy some desirable frequentist properties such as consistency and can be used to obtain asymptotically valid standard errors. In addition to these results we introduce two VB information criteria: the variational Akaike information criterion and the variational Bayesian information criterion. We show that variational Akaike information criterion is asymptotically equivalent to the frequentist Akaike information criterion and that the variational Bayesian information criterion is first order equivalent to the Bayesian information criterion in linear regression. These results motivate the potential use of the variational information criteria for more complex models. We support our theoretical results with numerical examples.

show abstract

Section: Introductionmentioning

confidence: 99%

On Variational Bayes Estimation and Variational Information Criteria for Linear Regression Models

You

Ormerod

Müller

2014

Aust. N. Z. J. Stat.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The MFVB methods we considered here are fast and versatile and can be easily extended to more complicated scenarios. For example, the methods allow arbitrary priors for the hyperparameters [57] and similar types of model with Gaussian responses [18,20,21]. Stewart [22] provides great examples of more elaborate models within the context of social sciences.…”

Section: Discussionmentioning

confidence: 99%

Variational methods for fitting complex Bayesian mixed effects models to health data

Lee

Wand

2015

Statistics in Medicine

View full text Add to dashboard Cite

We consider approximate inference methods for Bayesian inference to longitudinal and multilevel data within the context of health science studies. The complexity of these grouped data often necessitates the use of sophisticated statistical models. However, the large size of these data can pose significant challenges for model fitting in terms of computational speed and memory storage. Our methodology is motivated by a study that examines trends in cesarean section rates in the largest state of Australia, New South Wales, between 1994 and 2010. We propose a group-specific curve model that encapsulates the complex nonlinear features of the overall and hospital-specific trends in cesarean section rates while taking into account hospital variability over time. We use penalized spline-based smooth functions that represent trends and implement a fully mean field variational Bayes approach to model fitting. Our mean field variational Bayes algorithms allow a fast (up to the order of thousands) and streamlined analytical approximate inference for complex mixed effects models, with minor degradation in accuracy compared with the standard Markov chain Monte Carlo methods.

show abstract

“…We say “almost” since, courtesy of various recurrence formulae, there are some closely related ratios that can be written in terms of the Table ratios. For example arises in Algorithm 5 of Neville et al ().…”

Section: Special Function Ratios Arising In Bayesian Computingmentioning

confidence: 99%

“…This ratio involving the exponential integral function of order 1: arises in Neville et al (), where mean field variational Bayesian approximate inference for sparse signal regression with a Horseshoe prior distribution (Carvalho et al, ) is treated.…”

Section: Special Function Ratios Arising In Bayesian Computingmentioning

confidence: 99%

“…Another continuous sparse signal shrinkage distribution treated in Neville et al () is the Normal‐Exponential‐Gamma distribution (Griffin & Brown, ). The mean field variational Bayes updates for the single auxiliary variable representation of the Normal‐Exponential‐Gamma distribution involves repeated computation of the parabolic cylinder function ratio D − ν − 2 ( x ) ∕ D − ν − 1 ( x ) with ν > 0.…”

Section: Special Function Ratios Arising In Bayesian Computingmentioning

confidence: 99%

See 1 more Smart Citation

Continued fraction enhancement of Bayesian computing

Wand

Ormerod

2012

Stat

Self Cite

View full text Add to dashboard Cite

Soft tissue reactions to materials in joint prostheses include discoloration, fibrosis, florid histiocytic reaction, and granulomatous inflammation with foreign body giant cell reaction. Clinical manifestations include pain and swelling. We report a case of temporomandibular joint Proplast-Teflon prosthesis, followed by the development of large cell lymphoma in the left parotid gland 10 years after joint replacement. While it is unclear whether the implant directly contributed to the development of lymphoma, this association has not been previously documented, prompting this report.

show abstract

Mean field variational Bayes for continuous sparse signal shrinkage: Pitfalls and remedies

Cited by 43 publications

References 19 publications

On Variational Bayes Estimation and Variational Information Criteria for Linear Regression Models

On Variational Bayes Estimation and Variational Information Criteria for Linear Regression Models

Variational methods for fitting complex Bayesian mixed effects models to health data

Continued fraction enhancement of Bayesian computing

Contact Info

Product

Resources

About