Computing the Partition Function of the Sherrington-Kirkpatrick Model is Hard on Average

Gamarnik, David; Kızıldağ, Eren C.

doi:10.1109/isit44484.2020.9174373

Cited by 4 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(It is worth noting though that a notable exception to this is when the problem exhibits random self-reducibility, see e.g. [GK21b] for such a hardness result regarding a spin glass model, conditional on a weaker assumption P = #P .) Nevertheless, a very fruitful (and still active) line of research proposed certain forms of rigorous evidences of algorithmic hardness for such average-case problems.…”

Section: Background and Related Workmentioning

confidence: 99%

Algorithms and Barriers in the Symmetric Binary Perceptron Model

Gamarnik¹,

Kızıldağ²,

Perkins³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The binary (or Ising) perceptron is a toy model of a single-layer neural network and can be viewed as a random constraint satisfaction problem with a high degree of connectivity. The model and its symmetric variant, the symmetric binary perceptron (SBP), have been studied widely in statistical physics, mathematics, and machine learning.The SBP exhibits a dramatic statistical-to-computational gap: the densities at which known efficient algorithms find solutions are far below the threshold for the existence of solutions. Furthermore, the SBP exhibits a striking structural property: at all positive constraint densities almost all of its solutions are 'totally frozen' singletons separated by large Hamming distance [PX21,ALS21b]. This suggests that finding a solution to the SBP may be computationally intractable. At the same time, however, the SBP does admit polynomial-time search algorithms at low enough densities. A conjectural explanation for this conundrum was put forth in [BDVLZ20]: efficient algorithms succeed in the face of freezing by finding exponentially rare clusters of large size. However, it was discovered recently that such rare large clusters exist at all subcritical densities, even at those well above the limits of known efficient algorithms [ALS21a]. Thus the driver of the statisticalto-computational gap exhibited by this model remains a mystery.In this paper, we conduct a different landscape analysis to explain the algorithmic tractability of this problem. We show that at high enough densities the SBP exhibits the multi Overlap Gap Property (m−OGP), an intricate geometrical property known to be a rigorous barrier for large classes of algorithms. Our analysis shows that the m−OGP threshold (a) is well below the satisfiability threshold; and (b) matches the best known algorithmic threshold up to logarithmic factors as m → ∞. We then prove that the m−OGP rules out the class of stable algorithms for the SBP above this threshold. We conjecture that the m → ∞ limit of the m-OGP threshold marks the algorithmic threshold for the problem. Furthermore, we investigate the stability of known efficient algorithms for perceptron models and show that the Kim-Roche algorithm [KR98], devised for the asymmetric binary perceptron, is stable in the sense we consider.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Algorithms and Barriers in the Symmetric Binary Perceptron Model

Gamarnik¹,

Kızıldağ²,

Perkins³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…First, in the worst case over the disorder G (p) , achieving any constant approximation ratio to the true optimum value is known to be quasi-NP hard even for degree 2 polynomials [ABE + 05, BBH + 12]. For the Sherrington-Kirkpatrick model with ξ(t) = t 2 /2 on the cube, it was recently shown to be NP-hard on average to compute the exact value of the partition function [GK21a]. Of course, these computational hardness results demand much stronger guarantees than the approximate optimization with high probability that we consider.…”

Section: Further Backgroundmentioning

confidence: 99%

Tight Lipschitz Hardness for Optimizing Mean Field Spin Glasses

Huang¹,

Sellke²

2021

Preprint

View full text Add to dashboard Cite

We study the problem of algorithmically optimizing the Hamiltonian H N of a spherical or Ising mixed p-spin glass. The maximum asymptotic value OPT of H N /N is characterized by a variational principle known as the Parisi formula, proved first by Talagrand and in more generality by Panchenko. Recently developed approximate message passing algorithms efficiently optimize H N /N up to a value ALG given by an extended Parisi formula, which minimizes over a larger space of functional order parameters. These two objectives are equal for spin glasses exhibiting a no overlap gap property. However, ALG < OPT can also occur, and no efficient algorithm producing an objective value exceeding ALG is known.We prove that for mixed even p-spin models, no algorithm satisfying an overlap concentration property can produce an objective larger than ALG with non-negligible probability. This property holds for all algorithms with suitably Lipschitz dependence on the disorder coefficients of H N . It encompasses natural formulations of gradient descent, approximate message passing, and Langevin dynamics run for bounded time and in particular includes the algorithms achieving ALG mentioned above. To prove this result, we substantially generalize the overlap gap property framework introduced by Gamarnik and Sudan to arbitrary ultrametric forbidden structures of solutions. Contents

show abstract

“…The latter problem is known to be in the #P complexity class, which subsumes NP. A similar reduction exists (15) for the problem of computing the partition function of a Sherrington-Kirkpatrick model described below, thus implying that computing partition functions for spinglass models is not possible by polynomial time algorithms unless P = #P. Another problem admitting average-case to worst-case reduction is the problem of finding a shortest vector in a lattice (16). The random to worst-case types of reduction described above would be ideal for our setting, as they would provide the most compelling evidence of hardness of these problems.…”

Section: In Search Of the Right Algorithmic Complexity Theorymentioning

confidence: 55%

The overlap gap property: A topological barrier to optimizing over random structures

Gamarnik

2021

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

The problem of optimizing over random structures emerges in many areas of science and engineering, ranging from statistical physics to machine learning and artificial intelligence. For many such structures, finding optimal solutions by means of fast algorithms is not known and often is believed not to be possible. At the same time, the formal hardness of these problems in the form of the complexity-theoretic NP-hardness is lacking. A new approach for algorithmic intractability in random structures is described in this article, which is based on the topological disconnectivity property of the set of pairwise distances of near-optimal solutions, called the Overlap Gap Property. The article demonstrates how this property 1) emerges in most models known to exhibit an apparent algorithmic hardness; 2) is consistent with the hardness/tractability phase transition for many models analyzed to the day; and, importantly, 3) allows to mathematically rigorously rule out a large class of algorithms as potential contenders, specifically the algorithms that exhibit the input stability (insensitivity).

show abstract

Computing the Partition Function of the Sherrington-Kirkpatrick Model is Hard on Average

Cited by 4 publications

References 23 publications

Algorithms and Barriers in the Symmetric Binary Perceptron Model

Algorithms and Barriers in the Symmetric Binary Perceptron Model

Tight Lipschitz Hardness for Optimizing Mean Field Spin Glasses

The overlap gap property: A topological barrier to optimizing over random structures

Contact Info

Product

Resources

About