EMVS: The EM Approach to Bayesian Variable Selection

Ročková, Veronika; George, Edward I.

doi:10.1080/01621459.2013.869223

Cited by 233 publications

(310 citation statements)

References 25 publications

Supporting

Mentioning

308

Contrasting

Order By: Relevance

“…The sslasso retains the advantages of two popular methods for high-dimensional data analysis (V. Ročková and E. I. George, unpublished results), i.e., Bayesian variable selection McCulloch 1993, 1997;Chipman 1996;Chipman et al 2001;Ročková and George 2014), and the penalized lasso (Tibshirani 1996(Tibshirani , 1997Hastie et al 2015), and bridges these two methods into one unifying framework. Similar to the lasso, the proposed method can shrink many coefficients exactly to zero, thus automatically achieving variable selection and yielding easily interpretable results.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

The Spike-and-Slab Lasso Generalized Linear Models for Prediction and Associated Genes Detection

et al. 2017

View full text Add to dashboard Cite

Large-scale "omics" data have been increasingly used as an important resource for prognostic prediction of diseases and detection of associated genes. However, there are considerable challenges in analyzing high-dimensional molecular data, including the large number of potential molecular predictors, limited number of samples, and small effect of each predictor. We propose new Bayesian hierarchical generalized linear models, called spike-and-slab lasso GLMs, for prognostic prediction and detection of associated genes using large-scale molecular data. The proposed model employs a spike-and-slab mixture double-exponential prior for coefficients that can induce weak shrinkage on large coefficients, and strong shrinkage on irrelevant coefficients. We have developed a fast and stable algorithm to fit large-scale hierarchal GLMs by incorporating expectation-maximization (EM) steps into the fast cyclic coordinate descent algorithm. The proposed approach integrates nice features of two popular methods, i.e., penalized lasso and Bayesian spike-and-slab variable selection. The performance of the proposed method is assessed via extensive simulation studies. The results show that the proposed approach can provide not only more accurate estimates of the parameters, but also better prediction. We demonstrate the proposed procedure on two cancer data sets: a well-known breast cancer data set consisting of 295 tumors, and expression data of 4919 genes; and the ovarian cancer data set from TCGA with 362 tumors, and expression data of 5336 genes. Our analyses show that the proposed procedure can generate powerful models for predicting outcomes and detecting associated genes. The methods have been implemented in a freely available R package BhGLM

show abstract

Section: Discussionmentioning

confidence: 99%

“…The mixture normal priors cannot shrink coefficients exactly to zero, and thus cannot automatically perform variable selection. Ročková and George (2014) developed an expectationmaximization (EM) algorithm to fit large-scale linear models with the mixture normal priors.…”

mentioning

confidence: 99%

The Spike-and-Slab Lasso Generalized Linear Models for Prediction and Associated Genes Detection

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Inducing a variant of "selective shrinkage" Rao, 2005, 2011;Rockova, 2015), posterior modes under the SSL prior are adaptively thresholded, smaller values shrunk to exact zero. This is in sharp contrast to spike-and-slab priors with a Gaussian spike (George and McCulloch, 1993;Rockova and George, 2014), whose non-sparse posterior modes must be thresholded for variable selection. The exact sparsity here is crucial for anchoring on interpretable factor orientations and thus alleviating identifiability issues.…”

Section: Infinite Factor Model With Spike-and-slab Lassomentioning

confidence: 99%

Fast Bayesian Factor Analysis via Automatic Rotations to Sparsity

Ročková

George

2016

Journal of the American Statistical Association

Self Cite

155

View full text Add to dashboard Cite

Rotational transformations have traditionally played a key role in enhancing the interpretability of factor analysis via post-hoc modifications of the factor model orientation. Regularization methods also serve to achieve this goal by prioritizing sparse loading matrices. In this work, we cross-fertilize these two paradigms within a unifying Bayesian framework. Our approach deploys intermediate factor rotations throughout the learning process, greatly enhancing the effectiveness of sparsity inducing priors. These automatic rotations to sparsity are embedded within a PXL-EM algorithm, a Bayesian variant of parameter-expanded EM for posterior mode detection. By iterating between soft-thresholding of small factor loadings and transformations of the factor basis, we obtain (a) dramatic accelerations, (b) robustness against poor initializations and (c) better oriented sparse solutions. For accurate recovery of factor loadings, we deploy a two-component refinement of the Laplace prior, the spike-and-slab LASSO prior. This prior is coupled with the Indian Buffet Process (IBP) prior to avoid the pre-specification of the factor cardinality. The ambient dimensionality is learned from the posterior, which is shown to reward sparse matrices. Our deployment of PXL-EM performs a dynamic posterior exploration, outputting a solution path indexed by a sequence of spike-and-slab priors. A companion criterion, motivated as an integral lower bound, is provided to effectively select the best recovery. The potential of the proposed procedure is demonstrated on both simulated and real high-dimensional data, which would render posterior simulation impractical.

show abstract

“…It is in this context, when there are large numbers of parameters relative to the sample size, that spike-and-slab priors have become popular. The seminal article for assessing covariates in this context is George and McCulloch (1993), and recent conceptual and computational advances, say from Scott and Berger (2010) and Ročková and George (2014), make the approach feasible in increasing large big-data contexts.…”

Section: The Potential Of Spike-and-slab Models In Psychologymentioning

confidence: 99%

Bayesian inference for psychology, part IV: parameter estimation and Bayes factors

2018

View full text Add to dashboard Cite

a This is a preprint of a manuscript submitted for publication with Psychonomic Bulletin and Review.In the psychological literature, there are two seemingly different approaches to inference: that from estimation of posterior intervals and that from Bayes factors. We provide an overview of each method and show that a salient difference is the choice of models. The two approaches as commonly practiced can be unified with a certain model specification, now popular in the statistics literature, called spike-and-slab priors. A spike-and-slab prior is a mixture of a null model, the spike, with an effect model, the slab. The estimate of the effect size here is a function of the Bayes factor, showing that estimation and model comparison can be unified. The salient difference is that common Bayes factor approaches provide for privileged consideration of theoretically useful parameter values, such as the value corresponding to the null hypothesis, while estimation approaches do not. Both approaches, either privileging the null or not, are useful depending on the goals of the analyst.Bayes factors | Bayesian estimation | Bayesian inference | ROPE | Hypothesis testing Bayesian analysis has become increasing popular in many fields including psychological science. There are many advantages to the Bayesian approach. Some champion its clear philosophical underpinnings where probability is treated as a statement of belief or information and the focus is on updating beliefs rationally in face of new data (de Finetti, 1974;Edwards, Lindman, & Savage, 1963). Others note the practical advantages-Bayesian analysis often provides a tractable means of solving difficult problems that remain intractable in more conventional frameworks (Gelman, Carlin, Stern, & Rubin, 2004). This practical advantage is especially pronounced in cognitive science where substantive models are designed to account for mental representation and processing. As a consequence, the models tend to be complex and nonlinear, and may include multiple sources of variation (Kruschke, 2011b;Lee & Wagenmakers, 2013;Rouder & Lu, 2005). Bayesian analysis, especially Bayesian nonlinear hierarchical modeling, has been particularly successful at providing straightforward analyses in these otherwise difficult settings (e.g., Rouder, Sun, Speckman, Lu, & Zhou, 2003;Vandekerckhove, Tuerlinckx, & Lee, 2011;Vandekerckhove, 2014).Bayesian analysis is not a unified field, and Bayesian statisticians disagree with one another in important ways (Senn, 2011). 1We highlight here two popular Bayesian approaches that may seem incompatible inasmuch as they provide different answers to what appears to be the same question. We discuss these approaches in the context of the simple problem where there is an experimental and control condition and we wish to characterize the evidence from the data for the presence or absence of an effect.In one approach, termed here the estimation approach, the difference between the conditions is represented by a parameter, 1 Perhaps such disagreements should be ...

show abstract

EMVS: The EM Approach to Bayesian Variable Selection

Cited by 233 publications

References 25 publications

The Spike-and-Slab Lasso Generalized Linear Models for Prediction and Associated Genes Detection

The Spike-and-Slab Lasso Generalized Linear Models for Prediction and Associated Genes Detection

Fast Bayesian Factor Analysis via Automatic Rotations to Sparsity

Bayesian inference for psychology, part IV: parameter estimation and Bayes factors

Contact Info

Product

Resources

About