Computing Low-Rank Approximations of Large-Scale Matrices with the Tensor Network Randomized SVD

Abstract

We propose a new algorithm for the computation of a singular value decomposition (SVD) low-rank approximation of a matrix in the Matrix Product Operator (MPO) format, also called the Tensor Train Matrix format. Our tensor network randomized SVD (TNrSVD) algorithm is an MPO implementation of the randomized SVD algorithm that is able to compute dominant singular values and their corresponding singular vectors. In contrast to the state-of-the-art tensorbased alternating least squares SVD (ALS-SVD) and modified alternating least squares SVD (MALS-SVD) matrix approximation methods, TNrSVD can be up to 17 times faster while achieving the same accuracy. In addition, our TNrSVD algorithm also produces accurate approximations in particular cases where both ALS-SVD and MALS-SVD fail to converge. We also propose a new algorithm for the fast conversion of a sparse matrix into its corresponding MPO form, which is up to 509 times faster than the standard Tensor Train SVD (TT-SVD) method while achieving machine precision accuracy. The efficiency and accuracy of both algorithms are demonstrated in numerical experiments.

Randomized Algorithms for Low-Rank Tensor Decompositions in the Tucker Format

“…To illustrate this, we report the runtime of the HOSVD, R-HOSVD, STHOSVD, and R-STHOSVD algorithms on X as the size of each dimension increases. For inputs, we fixed the target rank to be (5,5,5,5,5), the oversampling parameter as p = 5, and we used processing order ρ = [1,2,3,4,5] in the sequential algo- Figure 1. Left: Relative approximation error for 5-mode Hilbert tensor X ∈ R 25×25×25×25×25 defined in (6.1), with target rank (r, r, r, r, r) and oversampling parameter p = 5.…”

“…Right: Actual relative error for X from the R-HOSVD and R-STHOSVD algorithms compared to the calculated error bound as the target rank (r, r, r, r, r) increases. Both algorithms use oversampling parameter p = 5, and R-STHOSVD uses the processing order ρ = [1,2,3,4,5].…”

“…Each algorithm is run with target rank (5,5,5,5,5), and the randomized algorithms use oversampling parameter p = 5. The STHOSVD and R-STHOSVD algorithms use the processing order ρ = [1,2,3,4,5].…”

Matrix output extension of the tensor network Kalman filter with an application in MIMO Volterra system identification

“…In addition, the TT-cross algorithm is usually slow as it is likely that it needs to be restarted when the desired accuracy is not met. A recent alternative for the conversion of a sparse matrix to a tensor network is described in [3]. This newly-proposed algorithm converts a given sparse matrix directly into a tensor network without any dyadic decomposition.…”

“…The TT-cross approximation algorithm on the other hand is too slow in order to deploy it for real-time applications. Finally, the alternative matrix to tensor network conversion algorithm reported in [3] is best suited for sparse matrices, while C(t) will contain many nonzero entries. Fortunately, it is possible to derive an efficient algorithm that exploits the repeated Kronecker product structure to construct an exact tensor network representation of C(t).…”