Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors

Abstract

Abstract. The problem discussed in this paper is the symmetric best low multilinear rank approximation of third-order symmetric tensors. We propose an algorithm based on Jacobi rotations, for which symmetry is preserved at each iteration. Two numerical examples are provided indicating the need for such algorithms. An important part of the paper consists of proving that our algorithm converges to stationary points of the objective function. This can be considered an advantage of the proposed algorithm over existing symmetry-preserving algorithms in the literature.Key words. multilinear algebra, higher-order tensor, rank reduction, singular value decomposition, Jacobi rotation AMS subject classifications. 15A69, 65F99 DOI. 10.1137/11085743X1. Introduction. Higher-order tensors (three-way arrays) have been used as a tool in higher-order statistics [39,36,47,38] and independent component analysis (ICA) [13,14,19,9] for several decades. Other application areas include chemometrics, scientific computing, biomedical signal processing, image processing, and telecommunications. For an exhaustive list and references we refer to [46,34,32,8,12].Let us first consider the general low multilinear rank approximation of thirdorder tensors. The problem consists of finding the best approximation of a given tensor A ∈ R I1×I2×I3 , subject to a constraint on the multilinear rank of the approximation. The concept of multilinear rank was first introduced in [25,26] and is simply a generalization of the row and column rank of matrices to higher-order tensors. We define the problem more precisely in the next section.A closed-form solution of the best low multilinear rank approximation problem is not known. A generalization of the singular value decomposition (SVD) [23, sect. 2.5] called higher-order SVD (HOSVD) has been studied in [15]. A variation of this decomposition is know as the Tucker decomposition [49,50]. In general, truncation of the HOSVD leads to a good but not necessarily to the best low multilinear rank approximation. Recent iterative algorithms solving the problem include geometric Newton [21,29]

On approximate diagonalization of third order symmetric tensors by orthogonal transformations

“…We assume that x * p+1 → ζ 0 for simplicity. Now we prove that d i * ,j * (W (p) ) → 0 and ω i * ,j * (W (p) ) → 0 when p ∈ P tends to infinity, and thus get (i) by the continuity of (19).…”

“…We get that x * p+1 → 0, and eventually from the proof of Theorem 6.2: d i * ,j * (W (p) ) → 0 and ω i * ,j * (W (p) ) ≥ 0 when p ∈ P is large enough. Then we prove (ii) by the continuity of (19). (iii) Note that there exist a finite number of accumulation points and |x * k | ≤ 1 for any k > 1.…”

Globally Convergent Jacobi-Type Algorithms for Simultaneous Orthogonal Symmetric Tensor Diagonalization

“…However, choosing the maximal element in (8) requires a search over all the elements, which may take additional time. In [17], it was suggested to take ε 1. If we choose (i k , j k ) in the cyclic order and ε is very small, then it is natural to expect that the inequality (9) will be often satisfied, and thus the behavior of the algorithm will be very close to the behavior of the Jacobi-C algorithm.…”

“…Example 2. In [17], the best low multilinear rank approximation problem for 3rd order symmetric tensors was formulated as a special case of problem (2). In fact, based on [15,Theorem 4.1], the cost function has the form…”

A novel non-iterative algorithm for low-multilinear-rank tensor approximation

“…The first class encompasses iterative algorithms which typically address a (nonlinear) LS problem [6], [4], [7]. Methods of the second class are non-iterative and focus on finding a reasonable, but suboptimal, low-mrank approximation within a finite number of steps.…”