Spike and slab Bayesian sparse principal component analysis

Abstract

Sparse principal component analysis (PCA) is a popular tool for dimensional reduction of high-dimensional data. Despite its massive popularity, there is still a lack of theoretically justifiable Bayesian sparse PCA that is computationally scalable. A major challenge is choosing a suitable prior for the loadings matrix, as principal components are mutually orthogonal. We propose a spike and slab prior that meets this orthogonality constraint and show that the posterior enjoys both theoretical and computational advantages. Two computational algorithms, the PX-CAVI and the PX-EM algorithms, are developed. Both algorithms use parameter expansion to deal with the orthogonality constraint and to accelerate their convergence speeds. We found that the PX-CAVI algorithm has superior empirical performance than the PX-EM algorithm and two other penalty methods for sparse PCA. The PX-CAVI algorithm is then applied to study a lung cancer gene expression dataset. R package VBsparsePCA with an implementation of the algorithm is available on The Comprehensive R Archive Network.

Feature Selection and Damage Identification for Urban Railway Track Using Bayesian Globally Sparse Principal Component Analysis

“…To tackle such problems, sparse PCA was developed by involving an automatic selection of the appropriate model dimensionality with high-dimensional data; i.e., the principal components obtained by sparse PCA are linear combinations of only a few important variables, which facilitates the interpretation of data feature selection in practice. The main methods commonly used are the addition of a regularization term [19,20] and the use of promoting sparsity of a prior distribution for the coefficients of the projection matrix [21,22]. However, the sparseness patterns of the original variables for each principal component may be different, and thus we need to understand which original variables are active for interpreting each principal component separately.…”

“…For conventional sparse PCA [21,22], its loading matrix P would be expected to have few nonzero coefficients. However, each principal component usually selects different relevant variables (data columns) in data matrix X because we do not have the same sparseness pattern, i.e., they cannot be expressed as the linear combination of the same active variables in X.…”

“…For further investigation of the damage identification performance purpose, we also investigate the results of the state-of-the-art Bayesian sparse PCA presented in [21], in which the prior of the loading matrix is modeled as a spike-and-slab prior, to analyze the monitoring data for the track structure. Global sparseness can also be achieved by restricting the sparse mode to joint row sparsity.…”

“…These results imply that our method still produces excellent damage identification performance even when using data from different measurement areas for PCA analysis. Similar to Scenario 1, we also present the confusion matrices between the actual damage results and damage results identified by running Bayesian sparse PCA from [21] in Figure 12. Similar to the results in Figure 8, very few damaged data columns (6.67% and 13.33%) can be correctly identified, and this method cannot be applied for track damage identification.…”