An improved analysis and unified perspective on deterministic and randomized low rank matrix approximations

Abstract

We introduce a Generalized LU-Factorization (GLU) for low-rank matrix approximation. We relate this to past approaches and extensively analyze its approximation properties. The established deterministic guarantees are combined with sketching ensembles satisfying Johnson-Lindenstrauss properties to present complete bounds. Particularly good performance is shown for the sub-sampled randomized Hadamard transform (SRHT) ensemble. Moreover, the factorization is shown to unify and generalize many past algorithms. It also helps to explain the effect of sketching on the growth factor during Gaussian Elimination.

Fast and stable randomized low-rank matrix approximation

“…The bottom three algorithms all start from the same mathematical expression (1), but have important differences. Namely, Tropp17 and Clarkson The recent preprint by Demmel, Grigori and Rusciano [15] studies low-rank approximants from the perspective of the LU factorization, and derives an approximant that has an extra term of the form (Y T ) † M , where M ∈ R (r+ )×n in addition to (3). The authors show its accuracy is between that of GN and HMT; experiments suggest the accuracy improvement over GN is usually marginal, which is perhaps expected as the random matrix (Y T ) † may have nothing to do with the column space of A.…”

“…Here we analyze the approximation accuracy (or error) A − Âr , ignoring the effect of roundoff errors. While we focus on the HMT and generalized Nyström methods, our analysis can be applied to any method based on projection, which includes all algorithms in Table 1 but [15]. While results on HMT and (plain) GN can be found in the literature, we believe the analysis here is simpler than most, and treats many methods in a unified fashion.…”

“…The analysis here is inspired by an observation in[46, § 4], which is attributed to Ipsen. Other (lengthier) derivations in the literature include the original[30], and those based on the Schur complement[15],[37].…”