Stability of Low-Rank Tensor Representations and Structured Multilevel Preconditioning for Elliptic PDEs

Abstract

Folding grid value vectors of size 2 L into Lth order tensors of mode size 2 × · · · × 2, combined with low-rank representation in the tensor train format, has been shown to result in highly efficient approximations for various classes of functions. These include solutions of elliptic PDEs on nonsmooth domains or with oscillatory data. This tensor-structured approach is attractive because it leads to highly compressed, adaptive approximations based on simple discretizations. Standard choices of the underlying bases, such as piecewise multilinear finite elements on uniform tensor product grids, entail the well-known matrix ill-conditioning of discrete operators. We demonstrate that, for low-rank representations, the use of tensor structure itself additionally introduces representation ill-conditioning, a new effect specific to computations in tensor networks. We analyze the tensor structure of a BPX preconditioner for a second-order linear elliptic operator and construct an explicit tensor-structured representation of the preconditioner, with ranks independent of the number L of discretization levels. The straightforward application of the preconditioner yields discrete operators whose matrix conditioning is uniform with respect to the discretization parameter, but in decompositions that suffer from representation ill-conditioning. By additionally eliminating certain redundancies in the representations of the preconditioned discrete operators, we obtain reduced-rank decompositions that are free of both matrix and representation ill-conditioning. For an iterative solver based on soft thresholding of low-rank tensors, we obtain convergence and complexity estimates and demonstrate its reliability and efficiency for discretizations with up to 2 50 nodes in each dimension.

A Block-Sparse Tensor Train Format for Sample-Efficient High-Dimensional Polynomial Regression

“…A comparison of the relative errors to the exact solution are depicted in Figure 2. This example demonstrates the limitations of the ansatz space S 6 7 which is not able to exploit the low-rank structure of the function f. Using S 6 7,1 can partially remedy this problem as can be seen by the improved sample efficiency. But since S 6 7,1 4S 6 7 the final approximation error of S 6 7,1 can not deceed that of S 6 7 .…”

“…This example demonstrates the limitations of the ansatz space S 6 7 which is not able to exploit the low-rank structure of the function f. Using S 6 7,1 can partially remedy this problem as can be seen by the improved sample efficiency. But since S 6 7,1 4S 6 7 the final approximation error of S 6 7,1 can not deceed that of S 6 7 . One can see that the dense format T 1 (V 6 8 ) produces the best results but is quite unstable compared to the other ansatz classes.…”

“…This instability is a result of the non-convexity of the set T r (V d p ) and we observe that the chance of getting stuck in a local minimum increases when the rank r is reduced from 8 to 1. Finally, we want to address the peaks that are observable at M ≈ 500 samples for T 8 (V 6 8 ) and M ≈ 1716 samples for S 6 7 . For this recall that the approximation in S 6 7 amounts to solving a linear system which is underdetermined for M < 1716 samples and overdetermined for M > 1716 samples.…”

“…These allow for a straight-forward extension of SINDy [2] but can also encode additional structure as presented in [3]. The compressive capabilities of tensor networks originate from this ability to exploit additional structure like smoothness, locality or selfsimilarity and have hence been used in solving high-dimensional equations [4][5][6][7]. In the context of optimal control tensor train networks have been utilized for solving the Hamilton-Jacobi-Bellman equation in [8,9], for solving backward stochastic differential equations in [10] and for the calculation of stock options prices in [11,12].…”

Low-rank tensor approximation of singularly perturbed boundary value problems in one dimension

“…The second contribution of this paper is the adaptation of the preconditioner introduced in [1] to this singularly perturbed case. The straightforward application of classic solvers (DMRG [33], AMEn [8], etc.)…”