Least Squares Sparse Principal Component Analysis: A Backward Elimination Approach to Attain Large Loadings

Abstract

Sparse principal components analysis (SPCA) is a technique for finding principal components with a small number of non-zero loadings. Our contribution to this methodology is twofold. First we derive the sparse solutions that minimise the least squares criterion subject to sparsity requirements. Second, recognising that sparsity is not the only requirement for achieving simplicity, we suggest a backward elimination algorithm that computes sparse solutions with large loadings. This algorithm can be run without specifying the number of non-zero loadings in advance. It is also possible to impose the requirement that a minimum amount of variance be explained by the components. We give thorough comparisons with existing SPCA methods and present several examples using real datasets.

SIMPCA: a framework for rotating and sparsifying principal components

“…In LS SPCA [26] the sparse components are computed by adding a sparsity constraint to Pearson's minimisation of the approximation error (left-hand side term of Equation 2). Hence, for a given set of indices, the nonzero coefficients of the j-th LS SPCA component,…”

Using sparse categorical principal components to estimate asset indices: new methods with an application to rural southeast asia

“…Merola () developed least squares SPCA, a method in which the sparse components explain the maximum variance of all the variables in the set. The optimization problem is solved by a backward elimination algorithm or by projection.…”

“…The sparse components computed by most SPCA methods are simply the PCs of a small subset of the observed variables (Moghaddam et al, 2007). As a result the sparse PCs explain well the highly correlated variables in the selected subset but ignore the variance of the variables that are not included (see Merola, 2015, for a discussion). Since an optimal SPCA solution cannot be found in reasonable time, these methods differ in how suboptimal solutions to the problems are computed (see Trendafilov, 2014, for a review of these methods).…”

“…Since an optimal SPCA solution cannot be found in reasonable time, these methods differ in how suboptimal solutions to the problems are computed (see Trendafilov, 2014, for a review of these methods). Merola (2015Merola ( , 2018 developed least squares SPCA, a method in which the sparse components explain the maximum variance of all the variables in the set. The optimization problem is solved by a backward elimination algorithm or by projection.…”

“…We also apply least squares sparse PCA (SPCA: Merola, 2015) to the aspect scaled categorical variables to derive sparse principal components, which show the key drivers of variation across households using only a limited number of variables. This involves only a small loss of optimality while retaining the monotonicity constraints.…”