Smoothed analysis of tensor decompositions

Bhaskara, Aditya; Charikar, Moses; Moitra, Ankur; Vijayaraghavan, Aravindan

doi:10.1145/2591796.2591881

Cited by 96 publications

(83 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These algorithms are typically based on finding a rank-one decomposition of the empirical 3rd or 4th moment tensor of the mixture; they heavily use the special structure of these moments for Gaussian mixtures. One paper we highlight is [BCMV14], which also uses much higher moments of the distribution. They show that in the smoothed analysis setting, the ℓth moment tensor of the distribution has algebraic structure which can be algorithmically exploited to recover the means.…”

Section: Related Workmentioning

confidence: 99%

Mixture models, robustness, and sum of squares proofs

Hopkins

2018

Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

We use the Sum of Squares method to develop new efficient algorithms for learning wellseparated mixtures of Gaussians and robust mean estimation, both in high dimensions, that substantially improve upon the statistical guarantees achieved by previous efficient algorithms. Our contributions are:• Mixture models with separated means: We study mixtures of k distributions in d dimensions, where the means of every pair of distributions are separated by at least k ε . In the special case of spherical Gaussian mixtures, we give a (dk) O(1/ε 2 ) -time algorithm that learns the means assuming separation at least k ε , for any ε > 0. This is the first algorithm to improve on greedy ("single-linkage") and spectral clustering, breaking a long-standing barrier for efficient algorithms at separation k 1/4 .• Robust estimation: When an unknown (1−ε)-fraction of X 1 , . . . , X n are chosen from a sub-Gaussian distribution with mean µ but the remaining points are chosen adversarially, we give an algorithm recovering µ to error ε 1−1/t in time d O(t 2 ) , so long as sub-Gaussianness up to O(t) moments can be certified by a Sum of Squares proof. This is the first polynomial-time algorithm with guarantees approaching the information-theoretic limit for non-Gaussian distributions. Previous algorithms could not achieve error better than ε 1/2 .Both of these results are based on a unified technique. Inspired by recent algorithms of Diakonikolas et al. in robust statistics, we devise an SDP based on the Sum of Squares method for the following setting: given X 1 , . . . , X n ∈ R d for large d and n = poly(d) with the promise that a subset of X 1 , . . . , X n were sampled from a probability distribution with bounded moments, recover some information about that distribution.

show abstract

Section: Related Workmentioning

confidence: 99%

Mixture models, robustness, and sum of squares proofs

Hopkins

2018

Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing

View full text Add to dashboard Cite

show abstract

“…For a given matrix Φ, the K-rank τ (Φ) is defined as the largest k for which every n × k submatrix of Φ has its smallest singular value larger than 1/τ . In [25], it is shown that the τ -robust Krank is super-additive, implying that the K-rank τ of the Khatri-Rao product is strictly larger than individual K-rank τ s of the input matrices.…”

Section: B Related Workmentioning

confidence: 99%

On the Restricted Isometry of the Columnwise Khatri–Rao Product

Khanna

Murthy

2018

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

The columnwise Khatri-Rao product of two matrices is an important matrix type, reprising its role as a structured sensing matrix in many fundamental linear inverse problems. Robust signal recovery in such inverse problems is often contingent on proving the restricted isometry property (RIP) of a certain system matrix expressible as a Khatri-Rao product of two matrices. In this work, we analyze the RIP of a generic columnwise Khatri-Rao product matrix by deriving two upper bounds for its k th order Restricted Isometry Constant (k-RIC) for different values of k. The first RIC bound is computed in terms of the individual RICs of the input matrices participating in the Khatri-Rao product. The second RIC bound is probabilistic, and is specified in terms of the input matrix dimensions. We show that the Khatri-Rao product of a pair of m×n sized random matrices comprising independent and identically distributed subgaussian entries satisfies k-RIP with arbitrarily high probability, provided m exceeds O(k log n). Our RIC bounds confirm that the Khatri-Rao product exhibits stronger restricted isometry compared to its constituent matrices for the same RIP order. The proposed RIC bounds are potentially useful in the sample complexity analysis of several sparse recovery problems.

show abstract

“…16 Using tensoring to produce linearly independent vectors has also been studied recently in the context of real vectors [BCMV14].…”

Section: Random Submatrices Of E(m R)mentioning

confidence: 99%

Reed–Muller Codes for Random Erasures and Errors

Abbé

Shpilka

Wigderson

2015

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

This paper studies the parameters for which Reed-Muller (RM) codes over GF (2) can correct random erasures and random errors with high probability, and in particular when can they achieve capacity for these two classical channels. Necessarily, the paper also studies properties of evaluations of multi-variate GF (2) polynomials on random sets of inputs.For erasures, we prove that RM codes achieve capacity both for very high rate and very low rate regimes. For errors, we prove that RM codes achieve capacity for very low rate regimes, and for very high rates, we show that they can uniquely decode at about square root of the number of errors at capacity.The proofs of these four results are based on different techniques, which we find interesting in their own right. In particular, we study the following questions about E(m, r), the matrix whose rows are truth tables of all monomials of degree ≤ r in m variables. What is the most (resp. least) number of random columns in E(m, r) that define a submatrix having full column rank (resp. full row rank) with high probability? We obtain tight bounds for very small (resp. very large) degrees r, which we use to show that RM codes achieve capacity for erasures in these regimes.Our decoding from random errors follows from the following novel reduction. For every linear code C of sufficiently high rate we construct a new code C , also of very high rate, such that for every subset S of coordinates, if C can recover from erasures in S, then C can recover from errors in S. Specializing this to RM codes and using our results for erasures imply our result on unique decoding of RM codes at high rate.Finally, two of our capacity achieving results require tight bounds on the weight distribution of RM codes. We obtain such bounds extending the recent [KLP12] bounds from constant degree to linear degree polynomials.

show abstract

Smoothed analysis of tensor decompositions

Cited by 96 publications

References 31 publications

Mixture models, robustness, and sum of squares proofs

Mixture models, robustness, and sum of squares proofs

On the Restricted Isometry of the Columnwise Khatri–Rao Product

Reed–Muller Codes for Random Erasures and Errors

Contact Info

Product

Resources

About