Deterministic CUR for Improved Large-Scale Data Analysis: An Empirical Study

Abstract

Low-rank approximations which are computed from selected rows and columns of a given data matrix have attracted considerable attention lately. They have been proposed as an alternative to the SVD because they naturally lead to interpretable decompositions which was shown to be successful in application such as fraud detection, fMRI segmentation, and collaborative filtering. The CUR decomposition of large matrices, for example, samples rows and columns according to a probability distribution that depends on the Euclidean norm of rows or columns or on other measures of statistical leverage. At the same time, there are various deterministic approaches that do not resort to sampling and were found to often yield factorization of superior quality with respect to reconstruction accuracy. However, these are hardly applicable to large matrices as they typically suffer from high computational costs. Consequently, many practitioners in the field of data mining have abandon deterministic approaches in favor of randomized ones when dealing with today's large-scale data sets. In this paper, we empirically disprove this prejudice. We do so by introducing a novel, linear-time, deterministic CUR approach that adopts the recently introduced Simplex Volume Maximization approach for column selection. The latter has already been proven to be successful for NMF-like decompositions of matrices of billions of entries. Our exhaustive empirical study on more than 30 synthetic and real-world data sets demonstrates that it is also beneficial for CUR-like decompositions. Compared to other deterministic CUR-like methods, it provides comparable reconstruction quality but operates much faster so that it easily scales to matrices of billions of elements. Compared to sampling-based methods, it provides competitive reconstruction quality while staying in the same run-time complexity class.

A DEIM Induced CUR Factorization

“…Some approaches seek to maximize the volume of the decomposition [14, Figure 1: Comparison of singular vectors (left, scaled, in red) and DEIM-CUR columns (right, in blue) for a data set drawn from two multivariate normal distributions having different principal axes. 25]. Numerous other algorithms instead use leverage scores [5,10,22,28].…”

Literature survey on low rank approximation of matrices

“…The applicability of CUR decomposition in various fields can be found in [4,119,149]. The generalization of CUR decomposition to Tensors has been described in [21].…”

Insight on Effectiveness of Frequently Exercised PQ Classification Techniques

“…classifier based on statistical approach and classifier based on deterministic approach. Statistical approach [6] is used in a situation of availability for massive dataset while deterministic approach [7] is used for limited size of dataset. A good example of classifier based on statistical approach will be Support Vector Machine and Neural Network [8], while example for classifier for deterministic approach will be fuzzy logic and rule-based expert system [9].…”