Tensor Clustering with Planted Structures: Statistical Optimality and Computational Limits

Luo, Yuetian; Zhang, Anru

doi:10.48550/arxiv.2005.10743

Cited by 9 publications

(15 citation statements)

References 67 publications

(130 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, the minimal requirements on SNR (often refereed to as the computational limits) for guaranteeing the existence of a polynomial-time initialization algorithm are generally unknown for most low-rank tensor models, even for the simple tensor PCA model. Interested readers are suggested to refer (Zhang and Xia, 2018;Luo and Zhang, 2020;Hopkins et al, 2015;Dudeja and Hsu, 2020;Arous et al, 2018) for the discussions on the computational hardness of tensor PCA. The computational limits are probably more difficult to study under generalized low-rank tensor models.…”

Section: Discussionmentioning

confidence: 99%

“…An mth-order tensor is a multilinear array with m ways, e.g., matrices are second order tensors. These multi-way structures often emerge when, to name a few, information features are collected from distinct domains Han et al, 2020;Bi et al, 2018;Zhang et al, 2020b;Wang and Zeng, 2019), the multi-relational interactions or higher-order interactions of entities are present (Ke et al, 2019;Jing et al, 2020;Kim et al, 2017;Paul and Chen, 2020;Luo and Zhang, 2020;Wang and Li, 2020;Ghoshdastidar and Dukkipati, 2017;Pensky and Zhang, 2019), or the higher-order moments of data are explored Sun et al, 2017;Hao et al, 2020). There is an increasing demand for effective methods to analyze large and complex tensorial datasets.…”

Section: Introductionmentioning

confidence: 99%

“…General Low-rank plus Sparse Tensor ModelSuppose that we observe data D, which can be, for instance, simply a tensorial observation such as the binary adjacency tensor of a hypergraph network or multi-layer network(Ke et al, 2019;Jing et al, 2020;Kim et al, 2017;Paul and Chen, 2020;Luo and Zhang, 2020;Wang and Li, 2020); a real-valued tensor describing multi-dimensional observations(Han et al, 2020;Sun et al, 2017;Sun and Li, 2019;; or a collection of pairs of tensor covariate and real-valued response(Hao et al, 2020;Zhang et al, 2020a;. At the core of our model is the assumption that the observed D is sampled from a distribution characterized by a latent large tensor.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Generalized Low-rank plus Sparse Tensor Estimation by Fast Riemannian Optimization

Cai¹,

Li²,

Dong³

2021

Preprint

View full text Add to dashboard Cite

We investigate a generalized framework to estimate a latent low-rank plus sparse tensor, where the low-rank tensor often captures the multi-way principal components and the sparse tensor accounts for potential model mis-specifications or heterogeneous signals that are unexplainable by the low-rank part. The framework is flexible covering both linear and non-linear models, and can easily handle continuous or categorical variables. We propose a fast algorithm by integrating the Riemannian gradient descent and a novel gradient pruning procedure. Under suitable conditions, the algorithm converges linearly and can simultaneously estimate both the low-rank and sparse tensors. The statistical error bounds of final estimates are established in terms of the gradient of loss function. The error bounds are generally sharp under specific statistical models, e.g., the robust tensor PCA and the community detection in hypergraph networks with outlier vertices. Moreover, our method achieves non-trivial error bounds for heavy-tailed tensor PCA whenever the noise has a finite 2 + ε moment. We apply our method to analyze the international trade flow dataset and the statistician hypergraph co-authorship network, both yielding new and interesting findings.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Generalized Low-rank plus Sparse Tensor Estimation by Fast Riemannian Optimization

Cai¹,

Li²,

Dong³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…r pd{2 q) in tensor regression and least singular value requirement (λ{σ ě Opp d{4 r1{4 q) in tensor SVD match the start-of-the-arts in literature (see Table 1 for a comparison). Rigorous evidence has been established to show that the least singular value lower bounds in tensor SVD are essential for any polynomial-time algorithm to succeed (Zhang and Xia, 2018;Brennan and Bresler, 2020;Luo and Zhang, 2020b). Although no rigorous evidence has been established at present, it has been conjectured that the sample complexity Op ?…”

Section: Tensor Svdmentioning

confidence: 99%

Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical Optimality and Second-Order Convergence

Luo

Zhang²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we consider the estimation of a low Tucker rank tensor from a number of noisy linear measurements. The general problem covers many specific examples arising from applications, including tensor regression, tensor completion, and tensor PCA/SVD. We propose a Riemannian Gauss-Newton (RGN) method with fast implementations for low Tucker rank tensor estimation. Different from the generic (super)linear convergence guarantee of RGN in the literature, we prove the first quadratic convergence guarantee of RGN for low-rank tensor estimation under some mild conditions. A deterministic estimation error lower bound, which matches the upper bound, is provided that demonstrates the statistical optimality of RGN. The merit of RGN is illustrated through two machine learning applications: tensor regression and tensor SVD. Finally, we provide the simulation results to corroborate our theoretical findings.

show abstract

“…As sketched in [43, Appendix A], this class contains many popular and powerful frameworks, including local algorithms on graphs, power iteration, and approximate message passing [35,11,56,64,36]. In addition, a recent flurry of work has shown that for many average-case problems in high-dimensional statistics, including planted clique, sparse PCA, community detection, and tensor PCA, low degree polynomials are as powerful as the best polynomial-time algorithms known [55,54,53,7,61,34,22,15,63,70,8,16]. Thus, showing that low degree polynomial algorithms fail at some threshold provides evidence that all polynomial-time algorithms fail at that threshold.…”

Section: Introductionmentioning

confidence: 99%

The Algorithmic Phase Transition of Random $k$-SAT for Low Degree Polynomials

Bresler¹,

Huang²

2021

Preprint

View full text Add to dashboard Cite

Let Φ be a uniformly random k-SAT formula with n variables and m clauses. We study the algorithmic task of finding a satisfying assignment of Φ. It is known that a satisfying assignment exists with high probability at clause density m/n < 2 k log 2 − 1 2 (log 2 + 1) + o k (1), while the best polynomial-time algorithm known, the Fix algorithm of Coja-Oghlan [25], finds a satisfying assignment at the much lower clause density (1 − o k (1))2 k log k/k. This prompts the question: is it possible to efficiently find a satisfying assignment at higher clause densities?To understand the algorithmic threshold of random k-SAT, we study low degree polynomial algorithms, which are a powerful class of algorithms including Fix, Survey Propagation guided decimation (with bounded or mildly growing number of message passing rounds), and paradigms such as message passing and local graph algorithms. We show that low degree polynomial algorithms can find a satisfying assignment at clause density (1 − o k (1))2 k log k/k, matching Fix, and not at clause density (1 + o k (1))κ * 2 k log k/k, where κ * ≈ 4.911. This shows the first sharp (up to constant factor) computational phase transition of random k-SAT for a class of algorithms. Our proof establishes and leverages a new many-way overlap gap property tailored to random k-SAT.

show abstract

Tensor Clustering with Planted Structures: Statistical Optimality and Computational Limits

Cited by 9 publications

References 67 publications

Generalized Low-rank plus Sparse Tensor Estimation by Fast Riemannian Optimization

Generalized Low-rank plus Sparse Tensor Estimation by Fast Riemannian Optimization

Low-rank Tensor Estimation via Riemannian Gauss-Newton: Statistical Optimality and Second-Order Convergence

The Algorithmic Phase Transition of Random $k$-SAT for Low Degree Polynomials

Contact Info

Product

Resources

About