Pass-Efficient Randomized Algorithms for Low-Rank Matrix Approximation Using Any Number of Views

Abstract

This paper describes practical randomized algorithms for low-rank matrix approximation that accommodate any budget for the number of views of the matrix. The presented algorithms, which are aimed at being as pass efficient as needed, expand and improve on popular randomized algorithms targeting efficient low-rank reconstructions. First, a more flexible subspace iteration algorithm is presented that works for any views v ≥ 2, instead of only allowing an even v. Secondly, we propose more general and more accurate singlepass algorithms. In particular, we propose a more accurate memory efficient single-pass method and a more general single-pass algorithm which, unlike previous methods, does not require prior information to assure near peak performance. Thirdly, combining ideas from subspace and single-pass algorithms, we present a more passefficient randomized block Krylov algorithm, which can achieve a desired accuracy using considerably fewer views than that needed by a subspace or previously studied block Krylov methods. However, the proposed accuracy enhanced block Krylov method is restricted to large matrices that are either accessed a few columns or rows at a time. Recommendations are also given on how to apply the subspace and block Krylov algorithms when estimating either the dominant left or right singular subspace of a matrix, or when estimating a normal matrix, such as those appearing in inverse problems. Computational experiments are carried out that demonstrate the applicability and effectiveness of the presented algorithms. Motivation.The primary motivation for this study was to develop algorithms to speed up inverse methods used to estimate parameters in models describing subsurface flow in geothermal reservoirs [36,3]. Inverting models describing complex geophysical processes, such as fluid flow in the subsurface, frequently involves matching a large data set using highly parameterized computational models. Running the model commonly involves solving an expensive and nonlinear forward problem. Despite the possible nonlinearity of the forward problem, the link between the model parameters and simulated observations is often described in terms of a Jacobian matrix J ∈ R N d ×N m , which locally linearizes the relationship between the parameters and observations. The size of J is therefore determined by the (large) parameter and observation spaces. In this case, explicitly forming J is out of the question since at best it involves solving N m direct problems (linearized forward simulations) or N d adjoint problems (linearized backward simulations) [6,23,35,38]. Nevertheless, the information contained in J can be helpful for the purpose of inverting the model using nonlinear inversion methods such as a Gauss-Newton or Levenberg-Marquardt approach, and for quantifying uncertainty.Using adjoint simulation, direct simulation and randomized algorithms, the necessary information can be extracted from J without ever explicitly forming the large matrix J . Bjarkason et al. [3] showed that inversion of a nonlinea...

Pass-Efficient Randomized LU Algorithms for Computing Low-Rank Matrix Approximation

“…The accuracy of the PowerLU Algorithm 3.1 is the approximation error obtained from (3.2). Theorem 4.1 in [2] gives us an error bound for (3.2), which we state here as Theorem 3.1.…”

“…However, to increase the accuracy of the computation, we have to increment by 2 passes each time due to multiplication by A T A. In [2], the authors proposed the generalized randomized SVD algorithm, which allows any number of passes v ≥ 2 by using generalized randomized subspace iteration. A similar algorithm was proposed in [11].…”

“…In some cases, the input matrix is enormous, and accessing the matrix is very expensive. Randomized SVD algorithms that minimize accesses of the matrix have been proposed in [2]. For the extremely large matrices, single-pass (single access) algorithms have also been studied recently.…”

Randomized Algorithms for Computation of Tucker decomposition and Higher Order SVD (HOSVD)

“…From one prospective, randomized algorithms for low-rank approximation can be divided into the following two categories [20]:…”

Single-pass randomized QLP decomposition for low-rank approximation

“…As we know, the cost of data communication is often much higher than the algorithm itself. In order to reduce the cost of data communication, some single-pass algorithms have been proposed [17][18][19][20][21][22]. In this paper, based on the idea of single-pass, we extend the work of Wu and Xiang [23] to the single-pass randomized QLP decomposition for computing low-rank approximation, where two randomized algorithms are provided.…”