Structural Convergence Results for Approximation of Dominant Subspaces from Block Krylov Spaces

Abstract

This paper is concerned with approximating the dominant left singular vector space of a real matrix A of arbitrary dimension, from block Krylov spaces generated by the matrix AA T and the block vector AX. Two classes of results are presented. First are bounds on the distance, in the two and Frobenius norms, between the Krylov space and the target space. The distance is expressed in terms of principal angles. Second are quality of approximation bounds, relative to the best approximation in the Frobenius norm. For starting guesses X of full column-rank, the bounds depend on the tangent of the principal angles between X and the dominant right singular vector space of A. The results presented here form the structural foundation for the analysis of randomized Krylov space methods. The innovative feature is a combination of traditional Lanczos convergence analysis with optimal approximations via least squares problems. 1 but not vice versa. It is in this sense that dominant subspace approximations are harder.This paper. We consider block Krylov space methods for computing dominant left singular vector spaces of general rectangular matrices and we present structural, deterministic bounds on the quality of the subspaces, for essentially general starting guesses. The innovative feature is a fusion of eigenvalue and singular value technology: We combine a traditional Lanczos convergence analysis [38] with optimal approximations via least squares problems [10,11].Our long-term goal is to put randomized Krylov space approximations on a firm numerical footing. However, at this preliminary first step, we make a few idealized assumptions:1. The block Krylov spaces have maximal dimension.2. The analysis assumes exact arithmetic and does not address the implementation of numerically stable recursions. Future work will need to deal with the challenging issues of finite precision arithmetic and viable numerical implementations, including recursions, numerical stability, maintaining orthogonality, deflation, adaptation of block size, and restarting. Empirical evaluations will have to assess whether the bounds are tight enough to be informative in practice.Overview. We start with a brief summary of our contributions (Section 2), followed by a comparison to existing work (Section 3). Auxiliary results (Section 4) set the stage for the proof of the main Theorems (Sections 5, 6, 7, and Appendix A). We end the main part of the paper with a perspective on open problems (Section 8).2. Results. After setting the context (Section 2.1), we give a brief summary of our bounds for: The distance between the Krylov space and the dominant left singular space (Section 2.2); a particular dominant subspace approximation from the Krylov space (Section 2.3); and the polynomials appearing in the approximation (Section 2.4). We end this section with a discussion of options for bounding the distance between the initial guess and the dominant right singular vector space (Section 2.5).2.1. Setting. To approximate the dominant left singular vector subspace of...

Pass-Efficient Randomized Algorithms for Low-Rank Matrix Approximation Using Any Number of Views

“…As mentioned previously, if the 1-view approach is not suitably accurate, then more accurate methods such as Algorithm 2 should be considered. While Algorithm 2 is more pass efficient than classical iterative methods, which involve matrix vector multiplication, its pass efficiency can be improved by using a randomized block Krylov approach [12,21,31,34,39]. The next subsection gives a brief overview of the current state-of-the art for randomized block Krylov methods.…”

“…Furthermore, their analysis suggests that the block Krylov approach can achieve a desired accuracy using fewer matrix views than a standard randomized subspace iteration method and using less time. Recently, [12] presented a theoretical analysis of the randomized block Krylov method for the problem of approximating the subspace spanned by the dominant left-singular vectors of a matrix.…”

“…Following the work of [12,21,31,34,39], an approximate basis Q c for the range of A can be found as the basis of a Krylov subspace given by (7.1)…”

“…The following subsection shows that this strategy equates to applying a popular low-rank approximation method, meant for positive-semidefinite (psd) matrices, to the normal matrix J * J . Recent studies by [12,41] consider the accuracy of approximate principal subspaces generated by the standard randomized subspace [41] and block Krylov [12] methods.…”

“…Combining the above thoughts we arrive at Algorithm 9, which is a more flexible block Krylov method than that used by [12,21,31,34,39] and allows v ≥ 2. For an even number of views Algorithm 9 uses A ≈ Q c Q * c A otherwise it uses A ≈ AQ r Q * r .…”