Optimization landscape of Tucker decomposition

Frandsen, Abraham; Ge, Rong

doi:10.1007/s10107-020-01531-z

Cited by 8 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Low-rank tensor estimation with Tucker decomposition. [FG20] analyzed the landscape of Tucker decomposition for tensor factorization, and showed benign landscape properties with suitable regularizations. [GRY11,MHWG14] developed convex relaxation algorithms based on minimizing the nuclear norms of unfolded tensors for tensor regression, and similar approaches were developed in [HMGW15] for robust tensor completion.…”

Section: Additional Related Workmentioning

confidence: 99%

Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements

Tong¹,

Ma²,

Prater-Bennette³

et al. 2021

Preprint

View full text Add to dashboard Cite

Tensors, which provide a powerful and flexible model for representing multi-attribute data and multiway interactions, play an indispensable role in modern data science across various fields in science and engineering. A fundamental task is to faithfully recover the tensor from highly incomplete measurements in a statistically and computationally efficient manner. Harnessing the low-rank structure of tensors in the Tucker decomposition, this paper develops a scaled gradient descent (ScaledGD) algorithm to directly recover the tensor factors with tailored spectral initializations, and shows that it provably converges at a linear rate independent of the condition number of the ground truth tensor for two canonical problemstensor completion and tensor regression -as soon as the sample size is above the order of n 3/2 ignoring other parameter dependencies, where n is the dimension of the tensor. This leads to an extremely scalable approach to low-rank tensor estimation compared with prior art, which suffers from at least one of the following drawbacks: extreme sensitivity to ill-conditioning, high per-iteration costs in terms of memory and computation, or poor sample complexity guarantees. To the best of our knowledge, ScaledGD is the first algorithm that achieves near-optimal statistical and computational complexities simultaneously for low-rank tensor completion with the Tucker decomposition. Our algorithm highlights the power of appropriate preconditioning in accelerating nonconvex statistical estimation, where the iteration-varying preconditioners promote desirable invariance properties of the trajectory with respect to the underlying symmetry in low-rank tensor factorization.

show abstract

Section: Additional Related Workmentioning

confidence: 99%

Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements

Tong¹,

Ma²,

Prater-Bennette³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Song et al (2019) gave polynomial-time (1 + ε)-approximation algorithms for many types of low-rank tensor decompositions with respect to the Frobenius norm, including CP and Tucker decompositions. Frandsen & Ge (2022) showed that if a third-order tensor has an exact Tucker decomposition, then all local minima of an appropriately regularized loss landscape are globally optimal. Several works recently studied Tucker decomposition in streaming models (Traoré et al, 2019;Sun et al, 2020) and a sliding window model (Jang & Kang, 2021).…”

Section: Related Workmentioning

confidence: 99%

Approximately Optimal Core Shapes for Tensor Decompositions

Ghadiri¹,

Fahrbach²,

Fu³

et al. 2023

Preprint

View full text Add to dashboard Cite

This work studies the combinatorial optimization problem of finding an optimal core tensor shape, also called multilinear rank, for a size-constrained Tucker decomposition. We give an algorithm with provable approximation guarantees for its reconstruction error via connections to higher-order singular values. Specifically, we introduce a novel Tucker packing problem, which we prove is NPhard, and give a polynomial-time approximation scheme based on a reduction to the 2-dimensional knapsack problem with a matroid constraint. We also generalize our techniques to tree tensor network decompositions. We implement our algorithm using an integer programming solver, and show that its solution quality is competitive with (and sometimes better than) the greedy algorithm that uses the true Tucker decomposition loss at each step, while also running up to 1000x faster.

show abstract

“…It is also important to keep in mind that as the optimization problem is nonconvex, we cannot guarantee that a local optimum is a global one, see Reference 32.…”

Section: Tensor Concepts and Preliminariesmentioning

confidence: 99%

Spectral partitioning of large and sparse 3‐tensors using low‐rank tensor approximation

Eldén

Dehghan

2022

Numerical Linear Algebra App

View full text Add to dashboard Cite

The problem of partitioning a large and sparse tensor is considered, where the tensor consists of a sequence of adjacency matrices. Theory is developed that is a generalization of spectral graph partitioning. A best rank‐(2,2,λ) approximation is computed for λ=1,2,3, and the partitioning is computed from the orthogonal matrices and the core tensor of the approximation. It is shown that if the tensor has a certain reducibility structure, then the solution of the best approximation problem exhibits the reducibility structure of the tensor. Further, if the tensor is close to being reducible, then still the solution of the exhibits the structure of the tensor. Numerical examples with synthetic data corroborate the theoretical results. Experiments with tensors from applications show that the method can be used to extract relevant information from large, sparse, and noisy data.

show abstract

Optimization landscape of Tucker decomposition

Cited by 8 publications

References 18 publications

Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements

Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Estimation from Incomplete Measurements

Approximately Optimal Core Shapes for Tensor Decompositions

Spectral partitioning of large and sparse 3‐tensors using low‐rank tensor approximation

Contact Info

Product

Resources

About