The Overlap Gap Property in Principal Submatrix Recovery

Gamarnik, David; Jagannath, Aukosh; Sen, Subhabrata

doi:10.48550/arxiv.1908.09959

Cited by 8 publications

(10 citation statements)

References 71 publications

(89 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A consequence of the 2−OGP established in Theorem 2.2 (which holds for energy levels E n = n, ∈ 1 2 , 1 ) is the presence of a certain property, called a free energy well (FEW), in the landscape of the NPP. This property is known to be a rigorous barrier for a family of Markov Chain Monte Carlo (MCMC) methods [AGJ + 20] and has been previously employed for other average-case problems [GJS19,GZ19] to establish slow mixing of the Markov chain associated with the MCMC method and thus the failure of the method. We establish the presence of a FEW in the landscape of NPP in Theorem 3.3; and leverage this property in Theorem 3.4 to establish the failure of a very natural class of MCMC dynamics tailored for the NPP.…”

Section: Our Contributionsmentioning

confidence: 99%

Algorithmic Obstructions in the Random Number Partitioning Problem

Gamarnik,

Kızıldağ

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider the algorithmic problem of finding a near-optimal solution for the number partitioning problem (NPP). This problem appears in many practical applications, including the design of randomized controlled trials, multiprocessor scheduling, and cryptography; and is also of theoretical significance. The NPP possesses the so-called statistical-tocomputational gap: when its input X has distribution N (0, I n ), the optimal value of the NPP is Θ ( √ n2 −n ) w.h.p.; whereas the best polynomial-time algorithm achieves the objective value of only 2 −Θ(log 2 n) , w.h.p.In this paper, we initiate the study of the nature of this gap. Inspired by insights from statistical physics, we study the landscape of the NPP and establish the presence of the Overlap Gap Property (OGP), an intricate geometric property which is known to be a rigorous evidence of an algorithmic hardness for large classes of algorithms. By leveraging the OGP, we establish that (a) any sufficiently stable algorithm, appropriately defined, fails to find a near-optimal solution with energy below 2 −ω(n log −1/5 n) ; and (b) a very natural Markov Chain Monte Carlo dynamics fails for find near-optimal solutions. Our simulation results suggest that the state of the art algorithm achieving the value 2 −Θ(log 2 n) is indeed stable, but formally verifying this is left as an open problem.OGP regards the overlap structure of m−tuples of solutions achieving a certain objective value. When m is constant we prove the presence of OGP for the objective values of order 2 −Θ(n) , and the absence of it in the regime 2 −o(n) . Interestingly, though, by considering overlaps with growing values of m we prove the presence of the OGP up to the level 2 −ω( √ n log n) . Our proof of the failure of stable algorithms at values 2 −ω(n log −1/5 n) employs methods from Ramsey Theory from the extremal combinatorics, and is of independent interest.

show abstract

Section: Our Contributionsmentioning

confidence: 99%

Algorithmic Obstructions in the Random Number Partitioning Problem

Gamarnik,

Kızıldağ

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We first show that there is any 5. e.g. Planted Clique Conjecture (Berthet and Rigollet, 2013;Brennan et al, 2018;Gao et al, 2017), Statistical Query based lower bounds (Brennan et al, 2020;Dudeja and Hsu, 2021;Feldman and Kanade, 2012;Kearns, 1998), and Overlap Gap Property based analysis (Arous et al, 2020;Gamarnik and Zadik, 2017;Gamarnik et al, 2019) .…”

Section: Reduction To Testing Problemmentioning

confidence: 99%

On Support Recovery with Sparse CCA: Information Theoretic and Computational Limits

Laha¹,

Mukherjee²

2021

Preprint

View full text Add to dashboard Cite

In this paper we consider asymptotically exact support recovery in the context of high dimensional and sparse Canonical Correlation Analysis (CCA). Our main results describe four regimes of interest based on information theoretic and computational considerations. In regimes of "low" sparsity we describe a simple, general, and computationally easy method for support recovery, whereas in a regime of "high" sparsity, it turns out that support recovery is information theoretically impossible. For the sake of information theoretic lower bounds, our results also demonstrate a non-trivial requirement on the "minimal" size of the non-zero elements of the canonical vectors that is required for asymptotically consistent support recovery. Subsequently, the regime of "moderate" sparsity is further divided into two sub-regimes. In the lower of the two sparsity regimes, using a sharp analysis of a coordinate thresholding (Deshpande and Montanari, 2014) type method, we show that polynomial time support recovery is possible. In contrast, in the higher end of the moderate sparsity regime, appealing to the "Low Degree Polynomial" Conjecture , we provide evidence that polynomial time support recovery methods are inconsistent. Finally, we carry out numerical experiments to compare the efficacy of various methods discussed.

show abstract

“…Recall the map → R( ) has a Jacobian ≥ 1, is C 1 and has a well de ned C 1 inverse since we have assumed that it is regular. Thus integrating (44) and performing a change of variable (to get the second inequality) we obtain…”

Section: Overlap Concentration: Proof Of Inequality (27)mentioning

confidence: 99%

“…Some background and related work: In recent years, there has been much progress in understanding such spiked matrix models, which have played a crucial role in the analysis of threshold phenomena in high-dimensional statistical models for almost two decades, but most of this work has focused on standard settings, by which we mean problem settings where the distribution P X is xed independent of the problem dimension n. This means that the expected number of non-zero components of X, even if "small", will scale linearly with n. Early rigorous results found in [39] determined the location of the information theoretic phase transition point in a spiked covariance model using spectral methods, and [40,41] did the same for the Wigner case. More recently, the information theoretic limits and those of hypothesis testing have been derived, with the additional structure of sparse vectors, for large but nite sizes [42][43][44]. A lot of e orts have also been devoted to computational aspects of sparse PCA with many remarkable results [33,[43][44][45][46][47][48][49][50].…”

Section: Introduction and Settingmentioning

confidence: 99%

“…More recently, the information theoretic limits and those of hypothesis testing have been derived, with the additional structure of sparse vectors, for large but nite sizes [42][43][44]. A lot of e orts have also been devoted to computational aspects of sparse PCA with many remarkable results [33,[43][44][45][46][47][48][49][50]. The picture that has emerged is that the information theoretic and computational phase transition regimes are not on the same scale and that the computational-to-statistical gap diverges in the limit of vanishing sparsity.…”

Section: Introduction and Settingmentioning

confidence: 99%

See 1 more Smart Citation

All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation

Barbier¹,

Macris²,

Rush³

2020

Preprint

View full text Add to dashboard Cite

We determine statistical and computational limits for estimation of a rank-one matrix (the spike) corrupted by an additive gaussian noise matrix, in a sparse limit, where the underlying hidden vector (that constructs the rank-one matrix) has a number of non-zero components that scales sub-linearly with the total dimension of the vector, and the signal-to-noise ratio tends to in nity at an appropriate speed. We prove explicit low-dimensional variational formulas for the asymptotic mutual information between the spike and the observed noisy matrix and analyze the approximate message passing algorithm in the sparse regime. For Bernoulli and Bernoulli-Rademacher distributed vectors, and when the sparsity and signal strength satisfy an appropriate scaling relation, we nd all-or-nothing phase transitions for the asymptotic minimum and algorithmic mean-square errors. These jump from their maximum possible value to zero, at well de ned signal-tonoise thresholds whose asymptotic values we determine exactly. In the asymptotic regime the statistical-to-algorithmic gap diverges indicating that sparse recovery is hard for approximate message passing.Preprint. Under review.

show abstract

The Overlap Gap Property in Principal Submatrix Recovery

Cited by 8 publications

References 71 publications

Algorithmic Obstructions in the Random Number Partitioning Problem

Algorithmic Obstructions in the Random Number Partitioning Problem

On Support Recovery with Sparse CCA: Information Theoretic and Computational Limits

All-or-nothing statistical and computational phase transitions in sparse spiked matrix estimation

Contact Info

Product

Resources

About