Fundamental tensor operations for large-scale data analysis using tensor network formats

Abstract

We discuss extended definitions of linear and multilinear operations such as Kronecker, Hadamard, and contracted products, and establish links between them for tensor calculus. Then we introduce effective low-rank tensor approximation techniques including Candecomp/Parafac (CP), Tucker, and tensor train (TT) decompositions with a number of mathematical and graphical representations. We also provide a brief review of mathematical properties of the TT decomposition as a low-rank approximation technique. With the aim of breaking the curseof-dimensionality in large-scale numerical analysis, we describe basic operations on large-scale vectors, matrices, and high-order tensors represented by TT decomposition. The proposed representations can be used for describing numerical methods based on TT decomposition for solving large-scale optimization problems such as systems of linear equations and symmetric eigenvalue problems.

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

“…Basic tensor operations are summarized in Table 2.1, and illustrated in Figures 2.1-2.13. We refer to [43,119,128] for more detail regarding the basic notations and tensor operations. For convenience, general operations, such as vec(¨) or diag(¨), are defined similarly to the MATLAB syntax.…”

“…. i n,K n ) [128]. Furthermore, the nested (hierarchical) form of such a generalized Tucker decomposition leads to the Tree Tensor Networks State (TTNS) model [149] (see Figure 2.15 and Figure 2.18), with possibly a varying order of cores, which can be formulated as X = G 1 ; B (1) , B (2) , .…”

“…. , Q N u, then the following mathematical operations can be performed directly in the Tucker format 5 , which admits a significant reduction in computational costs [128,175,177]:…”

“…In practice, due to the high computational complexity of tensor contractions, especially for tensor networks with loops, this operation is often performed approximately [66,107,138,167]. A variant of Tensor Trace [128] for the case of the partial tensor selfcontraction considers a tensor A P R RˆI 1ˆI2ˆ¨¨¨ˆINˆR and yields a reducedorder tensor r…”

Real‐time iterative learning control‐two applications with time scales between years and nanoseconds

“…Because of the multidimensional structure of the trial data or controller parameters, respectively, complexity and storage demand can significantly be reduced by tensor structures such as Tensor Trains, Tucker, Hierarchical Tucker, or Canonical Polyadic (CP). 8 For many of these decomposition algorithms, implementations and standard tools are already available. [9][10][11] This paper is organized as follows.…”