On Bayesian principal component analysis

Šmı́dl, Václav; Quinn, Anthony

doi:10.1016/j.csda.2007.01.011

Cited by 39 publications

(33 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tipping and Bishop [21] showed that the result was equivalent to standard PCA under certain choices of hyperparameters, and generalized PPCA to incorporate priors on the weights. Šmídl and Quinn [19] extended this work to a full Bayesian treatment which included priors on both components and weights, and considered the use of appropriate priors on the components to enforce orthogonality. Beyond standard PCA, several authors have proposed additional priors to encourage sparsity or non-negativity on the components [3,18].…”

Section: Background and Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Deflation Method for Structured Probabilistic PCA

Khanna

Ghosh

Poldrack

et al. 2017

Proceedings of the 2017 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Modern treatments of structured Principal Component Analysis often focus on the estimation of a single component under various assumptions or priors, such as sparsity and smoothness, and then the procedure is extended to multiple components by sequential estimation interleaved with deflation. While prior work has highlighted the importance of proper deflation for ensuring the quality of the estimated components, to our knowledge, proposed techniques have only been developed and applied to non-probabilistic principal component analyses, and are not trivially extended to probabilistic analyses. This work introduces a novel, robust and efficient deflation method for Probabilistic Principal Component Analysis using tools recently developed for constrained probabilistic estimation via information projection. The components estimated using the proposed deflation regain some of the interpretability of classic PCA such as straightforward estimates of variance explained, while retaining the ability to incorporate rich prior structure. Moreover, sequential estimation allows for scaling probabilistic techniques to be at par with their deterministic counterparts. Experimental results on simulated data demonstrate the utility of the proposed deflation in terms of component recovery, and evaluation on neuroimaging data show both qualitative and quantitative improvements in the quality of the estimated components. We also present timing experiments on real data to illustrate the importance of sequential estimation with proper deflation for scalability.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

“…We refer to prior work in the Bayesian literature evaluated on data with dimensionality < O(10 3 ) e.g. [19] used data of dimensionality 35, while deterministic approaches are routinely applied to > O(10 4 ) data. Our proposal pushes the scalability of probabilistic methods on par with their deterministic counterparts.…”

Section: Introductionmentioning

confidence: 99%

A Deflation Method for Structured Probabilistic PCA

Khanna

Ghosh

Poldrack

et al. 2017

Proceedings of the 2017 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

show abstract

“…Proof: Since the covariance of vec(Z) is I, the covariance ofX is given by (17), where r = RR and c = CC . Conversely, if the covariance matrix ofX is separable as in (17), there exist C and R such that c = CC and r = RR . Then (16) holds with Z = C −1X R −1 .…”

Section: B Bilinear Transformation and Separable Covariancementioning

confidence: 99%

“…Note that a similar modeling assumption is also used in (13) for the PSOPCA model. Proposition 1: The use of the bilinear transformation (16) is equivalent to the assumption of a separable (Kronecker product) covariance matrix onX cov(vec(X)) = r ⊗ c (17) where r ∈ R d r ×d r and c ∈ R d c ×d c are the row and column covariance matrices ofX, respectively. Proof: Since the covariance of vec(Z) is I, the covariance ofX is given by (17), where r = RR and c = CC .…”

Section: B Bilinear Transformation and Separable Covariancementioning

confidence: 99%

“…Proposition 1: The use of the bilinear transformation (16) is equivalent to the assumption of a separable (Kronecker product) covariance matrix onX cov(vec(X)) = r ⊗ c (17) where r ∈ R d r ×d r and c ∈ R d c ×d c are the row and column covariance matrices ofX, respectively. Proof: Since the covariance of vec(Z) is I, the covariance ofX is given by (17), where r = RR and c = CC . Conversely, if the covariance matrix ofX is separable as in (17), there exist C and R such that c = CC and r = RR .…”

Section: B Bilinear Transformation and Separable Covariancementioning

confidence: 99%

See 1 more Smart Citation

Bilinear Probabilistic Principal Component Analysis

Zhao

Yu²,

Kwok

2012

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Abstract-Probabilistic principal component analysis (PPCA) is a popular linear latent variable model for multi-layer performing dimension reduction on 1-D data in a probabilistic manner. However, when used on 2-D data such as images, PPCA suffers from the curse of dimensionality due to the subsequently large number of model parameters. To overcome this problem, we propose in this paper a novel probabilistic model on 2-D data called bilinear PPCA (BPPCA). This allows the establishment of a closer tie between BPPCA and its nonprobabilistic counterpart. Moreover, two efficient parameter estimation algorithms for fitting BPPCA are also developed. Experiments on a number of 2-D synthetic and real-world data sets show that BPPCA is more accurate than existing probabilistic and nonprobabilistic dimension reduction methods.

show abstract

Bayesian Blind Source Separation with Unknown Prior Covariance

Tichý

Šmı́dl

2015

Latent Variable Analysis and Signal Separation

View full text Add to dashboard Cite

On Bayesian principal component analysis

Cited by 39 publications

References 10 publications

A Deflation Method for Structured Probabilistic PCA

A Deflation Method for Structured Probabilistic PCA

Bilinear Probabilistic Principal Component Analysis

Bayesian Blind Source Separation with Unknown Prior Covariance

Contact Info

Product

Resources

About