Adaptive Sampling and Fast Low-Rank Matrix Approximation

Deshpande, Amit; Vempala, Santosh

doi:10.1007/11830924_28

Cited by 136 publications

(196 citation statements)

References 13 publications

Supporting

Mentioning

192

Contrasting

Order By: Relevance

“…, a m ), and can be computed in time O(min{mn 2 , m 2 n}) using Singular Value Decomposition (SVD). Some recent work on p = 2 case [1,2,3,4,5,9,12], initiated by a result due to Frieze, Kannan, and Vempala [7], has focused on algorithms for computing a k-dimensional subspace that gives (1 + )-approximation to the optimum in time O(mn·poly(k, 1/ )), i.e., linear in the number of co-ordinates we store. Most of these algorithms, with the exception of [1,12], depend on subroutines that sample poly(k, 1/ ) points from given a 1 , a 2 , .…”

Section: Introductionmentioning

confidence: 99%

Sampling-based dimension reduction for subspace approximation

Deshpande

Varadarajan

2007

Proceedings of the Thirty-Ninth Annual ACM Symposium on Theory of Computing

Self Cite

View full text Add to dashboard Cite

We give a randomized bi-criteria algorithm for the problem of finding a k-dimensional subspace that minimizes the L p -error for given points, i.e., p-th root of the sum of p-th powers of distances to given points, for any p ≥ 1. Our algorithm runs in timeÕ mn · k 3 (k/ ) p+1 and produces a subset of sizeÕ k 2 (k/ ) p+1 from the given points such that, with high probability, the span of these points gives a (1 + )-approximation to the optimal k-dimensional subspace. We also show a dimension reduction type of result for this problem where we can efficiently find a subset of sizeÕ k p+3 + (k/ ) p+2 such that, with high probability, their span contains a k-dimensional subspace that gives (1 + )-approximation to the optimum. We prove similar results for the corresponding projective clustering problem where we need to find multiple k-dimensional subspaces.

show abstract

Section: Introductionmentioning

confidence: 99%

Sampling-based dimension reduction for subspace approximation

Deshpande

Varadarajan

2007

Proceedings of the Thirty-Ninth Annual ACM Symposium on Theory of Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…That is, one needs to compute the core tensor S appearing in (1.1) accordingly. The papers [1,5,7,12,14,19,22,30,34,36,37] essentially choose S in a particular way.…”

Section: 2)mentioning

confidence: 99%

Low-Rank Approximation of Tensors

Friedland

Tammali

2015

Numerical Algebra, Matrix Theory, Differential-Algebraic Equations and Control Theory

View full text Add to dashboard Cite

In many applications such as data compression, imaging or genomic data analysis, it is important to approximate a given tensor by a tensor that is sparsely representable. For matrices, i.e. 2-tensors, such a representation can be obtained via the singular value decomposition, which allows to compute best rank k-approximations. For very big matrices a low rank approximation using SVD is not computationally feasible. In this case different approximations are available. It seems that variants of the CURdecomposition are most suitable.For d-mode tensors T ∈ ⊗ d i=1 R n i , with d > 2, many generalizations of the singular value decomposition have been proposed to obtain low tensor rank decompositions. The most appropriate approximation seems to be best (r1, . . . , r d )-approximation, which maximizes the 2 norm of the projection of T on ⊗ d i=1 Ui, where Ui is an ri-dimensional subspace R n i . One of the most common methods is the alternating maximization method (AMM). It is obtained by maximizing on one subspace Ui, while keeping all other fixed, and alternating the procedure repeatedly for i = 1, . . . , d. Usually, AMM will converge to a local best approximation. This approximation is a fixed point of a corresponding map on Grassmannians. We suggest a Newton method for finding the corresponding fixed point. We also discuss variants of CUR-approximation method for tensors.The first part of the paper is a survey on low rank approximation of tensors. The second new part of this paper is a new Newton method for best (r1, . . . , r d )-approximation. We compare numerically different approximation methods.2000 Mathematics Subject Classification. 14M15, 15A18, 15A69, 65H10, 65K10.

show abstract

“…Specifically, they show that there exists k columns with which one can get a √ k + 1 relative error approximation in Frobenius norm, which is tight. Later, Deshpande and Vempala [6] provides an algorithm with two steps which yields a relative approximation in expectation: first, approximate the "volume sampling" introduced in [5] by successively choosing one column at each step with carefully chosen probabilities; then, choose…”

Section: Comparison To Related Workmentioning

confidence: 99%

Deterministic Sparse Column Based Matrix Reconstruction via Greedy Approximation of SVD

Çivril¹,

Magdon‐Ismail²

2008

Algorithms and Computation

View full text Add to dashboard Cite

Abstract. Given a matrix A ∈ R m×n of rank r, and an integer k < r, the top k singular vectors provide the best rank-k approximation to A. When the columns of A have specific meaning, it is desirable to find (provably) "good" approximations to A k which use only a small number of columns in A. Proposed solutions to this problem have thus far focused on randomized algorithms. Our main result is a simple greedy deterministic algorithm with guarantees on the performance and the number of columns chosen. Specifically, our greedy algorithm chooses c columns from A with c = Owhere Cgr is the matrix composed of the c columns, C + gr is the pseudoinverse of Cgr (CgrC + gr A is the best reconstruction of A from Cgr), and µ(A) is a measure of the coherence in the normalized columns of A. The running time of the algorithm is O(SV D(A k ) + mnc) where SV D(A k ) is the running time complexity of computing the first k singular vectors of A. To the best of our knowledge, this is the first deterministic algorithm with performance guarantees on the number of columns and a (1 + ) approximation ratio in Frobenius norm. The algorithm is quite simple and intuitive and is obtained by combining a generalization of the well known sparse approximation problem from information theory with an existence result on the possibility of sparse approximation. Tightening the analysis along either of these two dimensions would yield improved results.

show abstract

Adaptive Sampling and Fast Low-Rank Matrix Approximation

Cited by 136 publications

References 13 publications

Sampling-based dimension reduction for subspace approximation

Sampling-based dimension reduction for subspace approximation

Low-Rank Approximation of Tensors

Deterministic Sparse Column Based Matrix Reconstruction via Greedy Approximation of SVD

Contact Info

Product

Resources

About