Matrix approximation and projective clustering via volume sampling

Deshpande, Amit; Rademacher, Luis; Vempala, Santosh; Wang, Grant

doi:10.1145/1109557.1109681

Cited by 172 publications

(341 citation statements)

References 0 publications

Supporting

Mentioning

334

Contrasting

Order By: Relevance

“…To our knowledge, such a dimension reduction result is not known for the projective clustering problem for any p, including the cases p = 2 and p = ∞. Previous results for the cases p = 1, 2, ∞ [10,4,13] only showed the existence of such a subspace spanned by poly( ks ) points -the algorithm for finding the subspace enumerated all subsets of poly( ks ) points. Our dimension reduction result, combined with the recent fixed-dimensional result of [6], yields an O(mn · poly( s ) + m(log m) f (s/ ) ) time algorithm for the projective clustering problem with k = 1.…”

Section: Subspace Projective Clusteringmentioning

confidence: 89%

“…, a m ), and can be computed in time O(min{mn 2 , m 2 n}) using Singular Value Decomposition (SVD). Some recent work on p = 2 case [1,2,3,4,5,9,12], initiated by a result due to Frieze, Kannan, and Vempala [7], has focused on algorithms for computing a k-dimensional subspace that gives (1 + )-approximation to the optimum in time O(mn·poly(k, 1/ )), i.e., linear in the number of co-ordinates we store. Most of these algorithms, with the exception of [1,12], depend on subroutines that sample poly(k, 1/ ) points from given a 1 , a 2 , .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Sampling-based dimension reduction for subspace approximation

Deshpande

Varadarajan

2007

Proceedings of the Thirty-Ninth Annual ACM Symposium on Theory of Computing

Self Cite

View full text Add to dashboard Cite

We give a randomized bi-criteria algorithm for the problem of finding a k-dimensional subspace that minimizes the L p -error for given points, i.e., p-th root of the sum of p-th powers of distances to given points, for any p ≥ 1. Our algorithm runs in timeÕ mn · k 3 (k/ ) p+1 and produces a subset of sizeÕ k 2 (k/ ) p+1 from the given points such that, with high probability, the span of these points gives a (1 + )-approximation to the optimal k-dimensional subspace. We also show a dimension reduction type of result for this problem where we can efficiently find a subset of sizeÕ k p+3 + (k/ ) p+2 such that, with high probability, their span contains a k-dimensional subspace that gives (1 + )-approximation to the optimum. We prove similar results for the corresponding projective clustering problem where we need to find multiple k-dimensional subspaces.

show abstract

Section: Subspace Projective Clusteringmentioning

confidence: 89%

Section: Introductionmentioning

confidence: 99%

Sampling-based dimension reduction for subspace approximation

Deshpande

Varadarajan

2007

Proceedings of the Thirty-Ninth Annual ACM Symposium on Theory of Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, in ref. 13 not in general guarantee the return of an SPSD approximant when applied to an SPSD matrix. The same holds true for approaches motivated by numerical analysis; in recent work, the authors of ref.…”

Section: Proof Of Theoremmentioning

confidence: 99%

“…Though our goals and corresponding algorithms are quite different in their approach and scope of application, it is of interest to note that our Theorem 1 can in fact be viewed as a kernel-level version of a theorem of ref. 13, where a related notion termed volume sampling is employed for column selection. However, in ref.…”

Section: Proof Of Theoremmentioning

confidence: 99%

Spectral methods in machine learning and new strategies for very large datasets

Belabbas

Wolfe

2009

Proc. Natl. Acad. Sci. U.S.A.

109

107

View full text Add to dashboard Cite

Spectral methods are of fundamental importance in statistics and machine learning, because they underlie algorithms from classical principal components analysis to more recent approaches that exploit manifold structure. In most cases, the core technical problem can be reduced to computing a low-rank approximation to a positive-definite kernel. For the growing number of applications dealing with very large or high-dimensional datasets, however, the optimal approximation afforded by an exact spectral decomposition is too costly, because its complexity scales as the cube of either the number of training examples or their dimensionality. Motivated by such applications, we present here 2 new algorithms for the approximation of positive-semidefinite kernels, together with error bounds that improve on results in the literature. We approach this problem by seeking to determine, in an efficient manner, the most informative subset of our data relative to the kernel approximation task at hand. This leads to two new strategies based on the Nyström method that are directly applicable to massive datasets. The first of these-based on sampling-leads to a randomized algorithm whereupon the kernel induces a probability distribution on its set of partitions, whereas the latter approach-based on sorting-provides for the selection of a partition in a deterministic way. We detail their numerical implementation and provide simulation results for a variety of representative problems in statistical data analysis, each of which demonstrates the improved performance of our approach relative to existing methods.statistical data analysis | kernel methods | low-rank approximation S pectral methods hold a central place in statistical data analysis. Indeed, the spectral decomposition of a positive-definite kernel underlies a variety of classical approaches such as principal components analysis (PCA), in which a low-dimensional subspace that explains most of the variance in the data is sought; Fisher discriminant analysis, which aims to determine a separating hyperplane for data classification; and multidimensional scaling (MDS), used to realize metric embeddings of the data. Moreover, the importance of spectral methods in modern statistical learning has been reinforced by the recent development of several algorithms designed to treat nonlinear structure in data-a case where classical methods fail. Popular examples include isomap (1), spectral clustering (2), Laplacian (3) and Hessian (4) eigenmaps, and diffusion maps (5). Though these algorithms have different origins, each requires the computation of the principal eigenvectors and eigenvalues of a positive-definite kernel.Although the computational cost (in both space and time) of spectral methods is but an inconvenience for moderately sized datasets, it becomes a genuine barrier as data sizes increase and new application areas appear. A variety of techniques, spanning fields from classical linear algebra to theoretical computer science (6), have been proposed to trade off analysis precision ...

show abstract

“…Recently, the effort has been towards eliminating the additive term in the inequality thereby yielding a relative approximation in the form A − Π C A F ≤ (1 + ) A − A k F . Along these lines, Deshpande et al [5] first shows the existence of such approximations introducing a sampling technique related to the volume of the simplex defined by the column subsets of size k, without giving a polynomial time algorithm. Specifically, they show that there exists k columns with which one can get a √ k + 1 relative error approximation in Frobenius norm, which is tight.…”

Section: Comparison To Related Workmentioning

confidence: 99%

Deterministic Sparse Column Based Matrix Reconstruction via Greedy Approximation of SVD

Çivril¹,

Magdon‐Ismail²

2008

Algorithms and Computation

View full text Add to dashboard Cite

Abstract. Given a matrix A ∈ R m×n of rank r, and an integer k < r, the top k singular vectors provide the best rank-k approximation to A. When the columns of A have specific meaning, it is desirable to find (provably) "good" approximations to A k which use only a small number of columns in A. Proposed solutions to this problem have thus far focused on randomized algorithms. Our main result is a simple greedy deterministic algorithm with guarantees on the performance and the number of columns chosen. Specifically, our greedy algorithm chooses c columns from A with c = Owhere Cgr is the matrix composed of the c columns, C + gr is the pseudoinverse of Cgr (CgrC + gr A is the best reconstruction of A from Cgr), and µ(A) is a measure of the coherence in the normalized columns of A. The running time of the algorithm is O(SV D(A k ) + mnc) where SV D(A k ) is the running time complexity of computing the first k singular vectors of A. To the best of our knowledge, this is the first deterministic algorithm with performance guarantees on the number of columns and a (1 + ) approximation ratio in Frobenius norm. The algorithm is quite simple and intuitive and is obtained by combining a generalization of the well known sparse approximation problem from information theory with an existence result on the possibility of sparse approximation. Tightening the analysis along either of these two dimensions would yield improved results.

show abstract

Matrix approximation and projective clustering via volume sampling

Cited by 172 publications

References 0 publications

Sampling-based dimension reduction for subspace approximation

Sampling-based dimension reduction for subspace approximation

Spectral methods in machine learning and new strategies for very large datasets

Deterministic Sparse Column Based Matrix Reconstruction via Greedy Approximation of SVD

Contact Info

Product

Resources

About