Online Principal Component Analysis in High Dimension: Which Algorithm to Choose?

Cardot, Hervé; Degras, David

doi:10.1111/insr.12220

Cited by 84 publications

(61 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One can use the divide and conquer approach presented in subsection A to compute first the matrix M h = X T h Y h and then evaluate the SVD of this matrix. We present here an alternative approach [58] by considering an incremental version of the SVD.…”

Section: Incremental Svd When N Is Largementioning

confidence: 99%

PLS for Big Data: A unified parallel algorithm for regularised group PLS

Micheaux¹,

Liquet²,

Sutton³

2019

Statist. Surv.

View full text Add to dashboard Cite

Partial Least Squares (PLS) methods have been heavily exploited to analyse the association between two blocs of data. These powerful approaches can be applied to data sets where the number of variables is greater than the number of observations and in presence of high collinearity between variables. Different sparse versions of PLS have been developed to integrate multiple data sets while simultaneously selecting the contributing variables. Sparse modelling is a key factor in obtaining better estimators and identifying associations between multiple data sets. The cornerstone of the sparsity version of PLS methods is the link between the SVD of a matrix (constructed from deflated versions of the original matrices of data) and least squares minimisation in linear regression. We present here an accurate description of the most popular PLS methods, alongside their mathematical proofs. A unified algorithm is proposed to perform all four types of PLS including their regularised versions. Various approaches to decrease the computation time are offered, and we show how the whole procedure can be scalable to big data sets.

show abstract

Section: Incremental Svd When N Is Largementioning

confidence: 99%

PLS for Big Data: A unified parallel algorithm for regularised group PLS

Micheaux¹,

Liquet²,

Sutton³

2019

Statist. Surv.

View full text Add to dashboard Cite

show abstract

“…To assess the statistical accuracy of the algorithms considered we choose a metric that we refer to as the subspace error used previously by Cardot and Degras [13]. Given a matrix U ∈ R D×K with orthonormal columns, the orthogonal projector onto the range of U is P U ≡ UU .…”

Section: Performance Metric: Accuracymentioning

confidence: 99%

Efficient Principal Subspace Projection of Streaming Data Through Fast Similarity Matching

Giovannucci

Minden

Pehlevan

et al. 2018

2018 IEEE International Conference on Big Data (Big Data)

View full text Add to dashboard Cite

Big data problems frequently require processing datasets in a streaming fashion, either because all data are available at once but collectively are larger than available memory or because the data intrinsically arrive one data point at a time and must be processed online. Here, we introduce a computationally efficient version of similarity matching [1], a framework for online dimensionality reduction that incrementally estimates the top K-dimensional principal subspace of streamed data while keeping in memory only the last sample and the current iterate. To assess the performance of our approach, we construct and make public a test suite containing both a synthetic data generator and the infrastructure to test online dimensionality reduction algorithms on real datasets, as well as performant implementations of our algorithm and competing algorithms with similar aims. Among the algorithms considered we find our approach to be competitive, performing among the best on both synthetic and real data.

show abstract

“…Projecting V n+1 onto the closed convex cone of non negative operators would require to compute the eigenvalues of V n+1 which is time consuming in high dimension even if V n+1 is a rank one perturbation to V n (see Cardot and Degras (2015)). We consider the following simple approximation to this projection which consists in replacing in (13) the descent step γ n by a thresholded one,…”

Section: Efficient Recursive Algorithmsmentioning

confidence: 99%

Fast estimation of the median covariation matrix with application to online robust principal components analysis

Cardot

Godichon‐Baggioni

2016

TEST

Self Cite

View full text Add to dashboard Cite

International audienceThe geometric median covariation matrix is a robust multivariate indicator of dispersion which can be extended without any difficulty to functional data. We define estimators, based on recursive algorithms, that can be simply updated at each new observation and are able to deal rapidly with large samples of high dimensional data without being obliged to store all the data in memory. Asymptotic convergence properties of the recursive algorithms are studied under weak conditions. The computation of the principal components can also be performed online and this approach can be useful for online outlier detection. A simulation study clearly shows that this robust indicator is a competitive alternative to minimum covariance determinant when the dimension of the data is small and robust principal components analysis based on projection pursuit and spherical projections for high dimension data. An illustration on a large sample and high dimensional dataset consisting of individual TV audiences measured at a minute scale over a period of 24 hours confirms the interest of considering the robust principal components analysis based on the median covariation matrix. All studied algorithms are available in the R package Gmedian on CRAN

show abstract

Online Principal Component Analysis in High Dimension: Which Algorithm to Choose?

Cited by 84 publications

References 45 publications

PLS for Big Data: A unified parallel algorithm for regularised group PLS

PLS for Big Data: A unified parallel algorithm for regularised group PLS

Efficient Principal Subspace Projection of Streaming Data Through Fast Similarity Matching

Fast estimation of the median covariation matrix with application to online robust principal components analysis

Contact Info

Product

Resources

About