We describe two recently proposed randomized algorithms for the construction of low-rank approximations to matrices, and demonstrate their application (inter alia) to the evaluation of the singular value decompositions of numerically low-rank matrices. Being probabilistic, the schemes described here have a finite probability of failure; in most cases, this probability is rather negligible (10 ؊17 is a typical value). In many situations, the new procedures are considerably more efficient and reliable than the classical (deterministic) ones; they also parallelize naturally. We present several numerical examples to illustrate the performance of the schemes.matrix ͉ SVD ͉ PCA L ow-rank approximation of linear operators is ubiquitous in applied mathematics, scientific computing, numerical analysis, and a number of other areas. In this note, we restrict our attention to two classical forms of such approximations, the singular value decomposition (SVD) and the interpolative decomposition (ID). The definition and properties of the SVD are widely known; we refer the reader to ref. 1 for a detailed description. The definition and properties of the ID are summarized in Subsection 1.1 below.Below, we discuss two randomized algorithms for the construction of the IDs of matrices. Algorithm I is designed to be used in situations where the adjoint A* of the m ϫ n matrix A to be decomposed can be applied to arbitrary vectors in a ''fast'' manner, and has CPU time requirements typically proportional to k⅐C A* ϩ k⅐m ϩ k 2 ⅐n, where k is the rank of the approximating matrix, and C A* is the cost of applying A* to a vector. Algorithm II is designed for arbitrary matrices, and its CPU time requirement is typically proportional to m⅐n⅐log(k) ϩ k 2 ⅐n. We also describe a scheme converting the ID of a matrix into its SVD for a cost proportional to k 2 ⅐(m ϩ n).Space constraints preclude us from reviewing the extensive literature on the subject; for a detailed survey, we refer the reader to ref. 2. Throughout this note, we denote the adjoint of a matrix A by A*, and the spectral (l 2 -operator) norm of A by ʈAʈ 2 ; as is well known, ʈAʈ 2 is the greatest singular value of A. Furthermore, we assume that our matrices have complex entries (as opposed to real); real versions of the algorithms under discussion are quite similar to the complex ones.This note has the following structure: Section 1 summarizes several known facts. Section 2 describes randomized algorithms for the low-rank approximation of matrices. Section 3 illustrates the performance of the algorithms via several numerical examples. Section 4 contains conclusions, generalizations, and possible directions for future research.
Section 1: PreliminariesIn this section, we discuss two constructions from numerical analysis, to be used in the remainder of the note. Subsection 1.1: Interpolative Decompositions. In this subsection, we define interpolative decompositions (IDs) and summarize their properties.The following lemma states that, for any m ϫ n matrix A of rank k, there exist an...