Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors

Abstract

We consider two problems that arise in machine learning applications: the problem of recovering a planted sparse vector in a random linear subspace and the problem of decomposing a random low-rank overcomplete 3-tensor. For both problems, the best known guarantees are based on the sum-of-squares method. We develop new algorithms inspired by analyses of the sum-of-squares method. Our algorithms achieve the same or similar guarantees as sum-ofsquares for these problems but the running time is significantly faster.For the planted sparse vector problem, we give an algorithm with running time nearly linear in the input size that approximately recovers a planted sparse vector with up to constant relative sparsity in a random subspace of R n of dimension up toΩ( √ n). These recovery guarantees match the best known ones of Barak, Kelner, and Steurer (STOC 2014) up to logarithmic factors.For tensor decomposition, we give an algorithm with running time close to linear in the input size (with exponent ≈ 1.086) that approximately recovers a component of a random 3-tensor over R n of rank up toΩ(n 4/3 ). The best previous algorithm for this problem due to Ge and Ma (RANDOM 2015) works up to rankΩ(n 3/2 ) but requires quasipolynomial time.

Spectral Algorithms for Tensor Completion

“…This was motivated by ideas developed in [12] for the tensor decomposition problem. This only imposes the rank bound r d k 1 .…”

“…In a certain random tensor model, we show that this second method can succesfully estimate 3-tensors from n rd 3=2 revealed entries for d Ä r d 3=2 . In the design and analysis of this method we were inspired by some recent work [12] on the tensor decomposition problem.…”

“…Motivated by these gaps, in this section we develop a different completion algorithm for the case k D 3, which avoids unfolding and relies instead on a certain "contraction" of the tensor with itself. This was motivated by ideas developed in [12] for the tensor decomposition problem. Under a natural model of random symmetric low-rank tensors, we prove that in the regime d Ä r d 3=2 , our algorithm succeeds in completing the tensor based on n rd 3=2 observed entries.…”

Higher order analysis of gene correlations by tensor decomposition

“…In this part we give an algorithm that finds gene-components and gene-values in (1) with presence of very small spectral norm error. Our algorithm is inspired by orthogonal tensor decompositions proposed by Anandkumar et al [4] and Ma et al [11]. In general, we want to recover the gene-values λ k,m and genecomponents −−→ α k,m for M κ samples of cellular system by finding a certain orthogonal tensor decomposition of every symmetric tensor T (κ) ∈ IR N ⊗3 which is a 3 rd order tensor over IR N such that T (κ) = i,j,k T (κ) i,j,k ⊗ e i ⊗ e j ⊗ e k where e 1 , e 2 , .…”

“…Then we run log N power iteration on top eigenvector of matrix U in (3) and the final output would be − → α , the single component of the tensor. The algorithm succeeds with probability at least 1/ log n over the randomness of the algorithm when the ration between the largest and second largest eigenvalue of contracted tensor is at least 1 + 1/ log n. We can find full analysis of similar method is in [11]. To find the corresponding λ κ,m we contract tensor T (κ) in all 3 ways by the component.…”