Optimization on the Hierarchical Tucker manifold – Applications to tensor completion

Abstract

MSC:In this work, we develop an optimization framework for problems whose solutions are well-approximated by Hierarchical Tucker (HT) tensors, an efficient structured tensor format based on recursive subspace factorizations. By exploiting the smooth manifold structure of these tensors, we construct standard optimization algorithms such as Steepest Descent and Conjugate Gradient for completing tensors from missing entries. Our algorithmic framework is fast and scalable to large problem sizes as we do not require SVDs on the ambient tensor space, as required by other methods. Moreover, we exploit the structure of the Gramian matrices associated with the HT format to regularize our problem, reducing overfitting for high subsampling ratios. We also find that the organization of the tensor can have a major impact on completion from realistic seismic acquisition geometries. These samplings are far from idealized randomized samplings that are usually considered in the literature but are realizable in practical scenarios. Using these algorithms, we successfully interpolate large-scale seismic data sets and demonstrate the competitive computational scaling of our algorithms as the problem sizes grow.

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives

“…For example, [Uschmajew and Vandereycken, 2013] developed the manifold structure for the HT tensors, while Lubich et al [2013] developed the concept of dynamical low-rank approximation for both HT and TT formats. Moreover, Riemannian optimization in the Tucker and TT/HT formats has been successfully applied to large-scale tensor completion problems [Da Silva and Herrmann, 2013, Kasai and Mishra, 2015.…”

Iterative Methods Based on Soft Thresholding of Hierarchical Tensors

“…Examples are methods based on the alternating least squares approach and the density matrix renormalization group [31,51], as well as Riemannian optimization on fixed-rank tensor manifolds [13,36]. For further details and references concerning such methods, see also [28, §10] and [25].…”

Tensor completion via optimization on the product of matrix manifolds

“…Using this several algorithms have been considered in the literature [12], [13], [14] for numerically efficient inferencing using optimization on smooth manifolds. Similarly, under the t-SVD and using the notion of tubal-rank as defined below, we see that the 3-D tensors of fixed tubal-rank also form a smooth manifold.…”

“…For example, such scenarios typically arise in seismics where at a given location one typically records a temporal signal of certain length [14], [10]. In this case one is given observations X(i, j, :) at indices (i, j) 2 ⌦, i 2 {1, 2, ..., n 1 }, j 2 {1, 2, ..., n 2 }, where ⌦ denotes the sampling set.…”