Selection of non-zero loadings in sparse principal component analysis

Gajjar, Shriram; Külahçı, Murat; Palazoğlu, Ahmet

doi:10.1016/j.chemolab.2017.01.018

Cited by 26 publications

(22 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…At this moment, little is known about which model selection method(s) are suitable for regularized simultaneous component analysis: Cross-validation is a popular choice, but it is known that cross-validation methods tend to retain more variables than needed (Chen & Chen, 2008). Other model selection methods, such as Index of Sparseness (Gajjar et al, 2017; Trendafilov, 2014; Zou et al, 2006), stability selection (Meinshausen & Bühlmann, 2010), and AIC, BIC type methods (e.g., Chen & Chen, 2008; Croux,Filzmoser, & Fritz, 2013; Guo, James, Levina, Michailidis, & Zhu, 2010), may be considered as alternative methods for model selection. Furthermore, regularized SCA needs to be further extended to incorporate categorical data, which are often seen in social and behavioral research.…”

Section: Discussionmentioning

confidence: 99%

“…Studying the usefulness of such comprehensive procedures is much needed and deserves full attention in a separate article. Recently, Gu and Van Deun (2018) studied a few model selection methods for regularized SCA and found that a relatively lesser known, computationally efficient method, namely the Index of Sparseness (Gajjar et al, 2017; Trendafilov, 2014; Zou et al, 2006), outperformed cross-validation in terms of selecting the proper component loading structure. Thus, a comprehensive, yet computationally feasible model selection procedure for deciding R , λ L , and λ G based on the Index of Sparseness may be promising, but in this article we refrain from discussing it, because the procedure requires development and validation via, for example, simulation studies.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

RegularizedSCA: Regularized simultaneous component analysis of multiblock data in R

Deun

2018

Behav Res

View full text Add to dashboard Cite

This article introduces a package developed for R (R Core Team, 2017) for performing an integrated analysis of multiple data blocks (i.e., linked data) coming from different sources. The methods in this package combine simultaneous component analysis (SCA) with structured selection of variables. The key feature of this package is that it allows to (1) identify joint variation that is shared across all the data sources and specific variation that is associated with one or a few of the data sources and (2) flexibly estimate component matrices with predefined structures. Linked data occur in many disciplines (e.g., biomedical research, bioinformatics, chemometrics, finance, genomics, psychology, and sociology) and especially in multidisciplinary research. Hence, we expect our package to be useful in various fields.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

RegularizedSCA: Regularized simultaneous component analysis of multiblock data in R

Deun

2018

Behav Res

View full text Add to dashboard Cite

show abstract

“…Let # o denote the total number of zero loadings in . Then IS, according to Gajjar, Kulahci, and Palazoglu 28 and Trendafilov 29 , is…”

Section: Methodsmentioning

confidence: 99%

“…In this study, to identify a suitable variable selection method for regularized SCA, we examined the performance of six methods, including CV with “one-standard-error” rule 26 , stability selection 25 , repeated double cross-validation (rdCV) 27 , Index of Sparseness (IS) 28–30 , Bolasso with CV 31–33 , and a BIC criterion 34,35 . We chose CV with the “one-standard-error” rule, rdCV, IS, and Bolasso, because they had been used successfully in various applications of sparse PCA methods, including early recognition and disease prediction 36 , schizophrenia research 37 , epidemics 38 , cardiac research 39 , environmental research 40 , and psychometrics 41 .…”

Section: Introductionmentioning

confidence: 99%

Variable Selection in the Regularized Simultaneous Component Analysis Method for Multi-Source Data Integration

Schipper

Deun

2019

Sci Rep

View full text Add to dashboard Cite

Interdisciplinary research often involves analyzing data obtained from different data sources with respect to the same subjects, objects, or experimental units. For example, global positioning systems (GPS) data have been coupled with travel diary data, resulting in a better understanding of traveling behavior. The GPS data and the travel diary data are very different in nature, and, to analyze the two types of data jointly, one often uses data integration techniques, such as the regularized simultaneous component analysis (regularized SCA) method. Regularized SCA is an extension of the (sparse) principle component analysis model to the cases where at least two data blocks are jointly analyzed, which - in order to reveal the joint and unique sources of variation - heavily relies on proper selection of the set of variables (i.e., component loadings) in the components. Regularized SCA requires a proper variable selection method to either identify the optimal values for tuning parameters or stably select variables. By means of two simulation studies with various noise and sparseness levels in simulated data, we compare six variable selection methods, which are cross-validation (CV) with the “one-standard-error” rule, repeated double CV (rdCV), BIC, Bolasso with CV, stability selection, and index of sparseness (IS) - a lesser known (compared to the first five methods) but computationally efficient method. Results show that IS is the best-performing variable selection method.

show abstract

“…8 principal components are selected. In this study, the Index of Sparseness (IS) is adopted to determine the number of non-zero coefficients (NNZC) for each principal component [33]. Fig.…”

Section: An Illustrative Examplementioning

confidence: 99%

Sparse Kernel Principal Component Analysis via Sequential Approach for Nonlinear Process Monitoring

Guo

Gao

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Kernel principal component analysis (KPCA) has been widely used for nonlinear process monitoring. However, since the principal components are linear combinations of all kernel functions, traditional KPCA suffers from poor interpretation and high-computation cost. To address this problem, obtaining sparse coefficients in KPCA is of paramount importance, particularly for real-time process monitoring and large-scale processes. In this paper, a new sparse kernel principal component analysis via sequential approach, named SSKPCA is proposed for nonlinear process monitoring. We first incorporate elastic net regularization into the framework of KPCA to establish a modified optimization problem. Then, a sequential approach is employed to derive the solution. Different from the existing sparse KPCA method based on elastic net regularization, the extra computations associated with the kernel matrix, such as matrix inversion and matrix square root are avoided in the optimizing procedure for solving the modified optimization problem. Therefore, the proposed SSKPCA method is more efficient in numerical implementation. The SSKPCA-based T 2 and squared prediction error (Q) statistics are constructed for fault detection. Furthermore, the sensitivity analysis principle is adopted for fault identification. A comparative study of Tennessee Eastman Process (TEP) is carried out to illustrate the ability and efficiency of the proposed SSKPCA-based nonlinear process monitoring method. INDEX TERMS Process monitoring, kernel-based methods, principal component analysis, sparsity, sequential approach.

show abstract

Selection of non-zero loadings in sparse principal component analysis

Cited by 26 publications

References 31 publications

RegularizedSCA: Regularized simultaneous component analysis of multiblock data in R

RegularizedSCA: Regularized simultaneous component analysis of multiblock data in R

Variable Selection in the Regularized Simultaneous Component Analysis Method for Multi-Source Data Integration

Sparse Kernel Principal Component Analysis via Sequential Approach for Nonlinear Process Monitoring

Contact Info

Product

Resources

About