Parameter Expansion for Data Augmentation

Liu, Jun S.; Wu, Ying

doi:10.2307/2669940

Cited by 111 publications

(166 citation statements)

References 0 publications

Supporting

Mentioning

166

Contrasting

Order By: Relevance

“…To address these issues, Ghosh & Dunson (2009) use parameter expansion (Liu & Wu, 1999;Gelman, 2006) to induce a heavy-tailed default prior distribution on the loading elements and propose an efficient Gibbs sampler.…”

Section: A Bhattacharya and D B Dunsonmentioning

confidence: 99%

Sparse Bayesian infinite factor models

Bhattacharya¹,

Dunson²

2011

Biometrika

353

486

View full text Add to dashboard Cite

SUMMARYWe focus on sparse modelling of high-dimensional covariance matrices using Bayesian latent factor models. We propose a multiplicative gamma process shrinkage prior on the factor loadings which allows introduction of infinitely many factors, with the loadings increasingly shrunk towards zero as the column index increases. We use our prior on a parameter-expanded loading matrix to avoid the order dependence typical in factor analysis models and develop an efficient Gibbs sampler that scales well as data dimensionality increases. The gain in efficiency is achieved by the joint conjugacy property of the proposed prior, which allows block updating of the loadings matrix. We propose an adaptive Gibbs sampler for automatically truncating the infinite loading matrix through selection of the number of important factors. Theoretical results are provided on the support of the prior and truncation approximation bounds. A fast algorithm is proposed to produce approximate Bayes estimates. Latent factor regression methods are developed for prediction and variable selection in applications with high-dimensional correlated predictors. Operating characteristics are assessed through simulation studies, and the approach is applied to predict survival times from gene expression data.

show abstract

Section: A Bhattacharya and D B Dunsonmentioning

confidence: 99%

Sparse Bayesian infinite factor models

Bhattacharya¹,

Dunson²

2011

Biometrika

353

486

View full text Add to dashboard Cite

show abstract

“…Perhaps the most well-known subclass of sandwich algorithms in the existing literature is the class of parameter-expansion data augmentation (PXDA) algorithms introduced by Liu and Wu [19]. If there exists a group structure G that acts on Y, then any probability distribution r supported on G would correspond to a PXDA algorithm, and we denote its X-chain operator by P r G .…”

Section: Improving Da Algorithms Using Haar Algorithmsmentioning

confidence: 99%

“…It is implicitly stated in Liu and Wu [19,Section.5] that the Haar algorithm based on a locally compact group G ′ is a special PXDA algorithm on G ′ that corresponds to the left Haar measure r = ν G ′ . (An alternative proof of the statement in the special case where G ′ is a discrete group is provided in the online supplement of this paper.…”

Section: Appendix A: Proof Of Propositionmentioning

confidence: 99%

See 1 more Smart Citation

Sandwich algorithms for Bayesian variable selection

Ghosh

Tan

2015

Computational Statistics & Data Analysis

View full text Add to dashboard Cite

Markov chain Monte Carlo (MCMC) algorithms have greatly facilitated the popularity of Bayesian variable selection and model averaging in problems with high-dimensional covariates where enumeration of the model space is infeasible. A variety of such algorithms have been proposed in the literature for sampling models from the posterior distribution in Bayesian variable selection. Ghosh and Clyde proposed a method to exploit the properties of orthogonal design matrices. Their data augmentation algorithm scales up the computation tremendously compared to traditional Gibbs samplers, and leads to the availability of RaoBlackwellized estimates of quantities of interest for the original non-orthogonal problem. The algorithm has excellent performance when the correlations among the columns of the design matrix are small, but empirical results suggest that moderate to strong multicollinearity leads to slow mixing. This motivates the need to develop a class of novel sandwich algorithms for Bayesian variable selection that improves upon the algorithm of Ghosh and Clyde. It is proved that the Haar algorithm with the largest group that acts on the space of models is the optimum algorithm, within the parameter expansion data augmentation (PXDA) class of sandwich algorithms. The result provides theoretical insight but using the largest group is computationally prohibitive so two new computationally viable sandwich algorithms are developed, which are inspired by the Haar algorithm, but do not necessarily belong to the class of PXDA algorithms. It is illustrated via simulation studies and real data analysis that several of the sandwich algorithms can offer substantial gains in the presence of multicollinearity.Keywords: Asymptotic variance, Data augmentation, Markov chain Monte Carlo, Multicollinearity, Rao-Blackwellization. * Corresponding author. Tel.: +1 319 335 0816.Email addresses: joyee-ghosh@uiowa.edu (Joyee Ghosh), aixin-tan@uiowa.edu (Aixin Tan) 1 The authors contributed equally. 2 Supplementary material includes details of algorithms, statement and proof of a lemma, and theoretical comparisons of sandwich algorithms in some examples.

show abstract

“…Of course the problem remains critical, for standard HMM theory does not allow for observation spaces of variable dimension. A possible solution can be the use of standard statistical techniques for the treatment of missing data [18] based on the EM algorithm. More interesting would be the learning of models based on hybrid systems composed by different HMMs each representing a state of occlusion.…”

Section: Towards Unsupervised Detectionmentioning

confidence: 99%