Variational calculus with sums of elementary tensors of fixed rank

Abstract

In this article we introduce a calculus of variations for sums of elementary tensors and apply it to functionals of practical interest. The survey provides all necessary ingredients for applying minimization methods in a general setting. The important cases of target functionals which are linear and quadratic with respect to the tensor product are discussed, and combinations of these functionals are presented in detail. As an example, we consider the solution of a linear system in structured tensor format. Moreover, we discuss the solution of an eigenvalue problem with sums of elementary tensors. This example can be viewed as a prototype of a constrained minimization problem. For the numerical treatment, we suggest a method which has the same order of complexity as the popular alternating least square algorithm and demonstrate the rate of convergence in numerical tests.

Iterative Solution of Large Sparse Systems of Equations

“…Further references to variational approaches are Espig-Hackbusch-Rohwedder-Schneider [125], Falcó-Nouy [126], Holtz-Rohwedder-Schneider [224], Mohlenkamp [284], Oseledets [299] and others cited in these papers. A diagonal matrix D is completely described by its diagonal entries.…”

On manifolds of tensors of fixed TT-rank

“…The canonical format, although surely scaling linearly with respect to the order d, the dimension n of the vector space and the canonical rank r , thus being ideal with regard to complexity, carries a lot of theoretical and practical drawbacks: The set of tensors of fixed canonical rank is not closed, and the existence of a best approximation is not guaranteed [8]. Although in some cases the approximation works quite well [10], optimization methods [11] often fail to converge as a consequence of uncontrollable redundancies in the parametrisation, and an actual computation of a low-rank approximation can thus be a numerically hazardous task. In contrast to this, the Tucker format, in essence corresponding to orthogonal projections into optimal subspaces of R n , still scales exponentially with the order d, only reducing the basis from n to the Tucker rank r .…”

Solution of linear systems in high spatial dimensions

“…Concerning the variational approach we refer to the following papers: Espig-Hackbusch-Rohwedder-Schneider [7], Falcó-Nouy [8], Holtz-Rohwedder-Schneider [21], Mohlenkamp [26], Osedelets [27] and others cited in these papers.…”