The Algorithmic Phase Transition of Random $k$-SAT for Low Degree Polynomials

Bresler, Guy; Huang, Brice

doi:10.48550/arxiv.2106.02129

Cited by 5 publications

(14 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This generalizes previous work [RV17] showing the same impossibility result for the more restricted class of local algorithms. Similarly, [BH21] used a multi-OGP to show that low degree polynomials fail to solve random k-SAT at a constant factor clause density above where algorithms are known to succeed.…”

Section: The Overlap Gap Property As a Barrier To Algorithmsmentioning

confidence: 99%

“…• Ladder OGP: several solutions, where the i-th solution (i ≥ 2) has medium "multi-overlap" with the first i − 1 solutions, for a problem-specific notion of multi-overlap of one solution with several solutions [Wei20,BH21].…”

Section: The Overlap Gap Property As a Barrier To Algorithmsmentioning

confidence: 99%

“…Specifically, in a "star" multi-OGP [RV17, GS17,GK21b] all the replicas are pairwise equidistant. For the "ladder" OGP implementations of [Wei20,BH21], the forbidden structure is defined by applying some stopping rule to choose a finite number of solutions from a "stably evolving" sequence of algorithmic outputs. In both settings it is possible that the resulting configuration is a star ultrametric with all pairwise nonzero distances equal.…”

Section: Necessity Of Full Branching Treesmentioning

confidence: 99%

See 2 more Smart Citations

Tight Lipschitz Hardness for Optimizing Mean Field Spin Glasses

Huang¹,

Sellke²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We study the problem of algorithmically optimizing the Hamiltonian H N of a spherical or Ising mixed p-spin glass. The maximum asymptotic value OPT of H N /N is characterized by a variational principle known as the Parisi formula, proved first by Talagrand and in more generality by Panchenko. Recently developed approximate message passing algorithms efficiently optimize H N /N up to a value ALG given by an extended Parisi formula, which minimizes over a larger space of functional order parameters. These two objectives are equal for spin glasses exhibiting a no overlap gap property. However, ALG < OPT can also occur, and no efficient algorithm producing an objective value exceeding ALG is known.We prove that for mixed even p-spin models, no algorithm satisfying an overlap concentration property can produce an objective larger than ALG with non-negligible probability. This property holds for all algorithms with suitably Lipschitz dependence on the disorder coefficients of H N . It encompasses natural formulations of gradient descent, approximate message passing, and Langevin dynamics run for bounded time and in particular includes the algorithms achieving ALG mentioned above. To prove this result, we substantially generalize the overlap gap property framework introduced by Gamarnik and Sudan to arbitrary ultrametric forbidden structures of solutions. Contents

show abstract

Section: The Overlap Gap Property As a Barrier To Algorithmsmentioning

confidence: 99%

Section: The Overlap Gap Property As a Barrier To Algorithmsmentioning

confidence: 99%

Section: Necessity Of Full Branching Treesmentioning

confidence: 99%

See 1 more Smart Citation

Tight Lipschitz Hardness for Optimizing Mean Field Spin Glasses

Huang¹,

Sellke²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is natural to ask whether better optimization algorithms can be constructed for this problem, or there is a fundamental computational barrier and indeed δ s (κ) = δ alg (κ). Such gaps are indeed quite common in discrete random CSPs [ACO08,COHH17,BH21]. We leave this question to future investigation.…”

Section: Summary Of Resultsmentioning

confidence: 89%

Tractability from overparametrization: The example of the negative perceptron

Montanari¹,

Zhong²,

Zhou³

2021

Preprint

View full text Add to dashboard Cite

In the negative perceptron problem we are given n data points (xi, yi), where xi is a d-dimensional vector and yi ∈ {+1, −1} is a binary label. The data are not linearly separable and hence we content ourselves to find a linear classifier with the largest possible negative margin. In other words, we want to find a unit norm vector θ that maximizes min i≤n yi θ, xi . This is a non-convex optimization problem (it is equivalent to finding a maximum norm vector in a polytope), and we study its typical properties under two random models for the data.We consider the proportional asymptotics in which n, d → ∞ with n/d → δ, and prove upper and lower bounds on the maximum margin κs(δ) or -equivalently-on its inverse function δs(κ). In other words, δs(κ) is the overparametrization threshold: for n/d ≤ δs(κ) − ε a classifier achieving vanishing training error exists with high probability, while for n/d ≥ δs(κ) + ε it does not. Our bounds on δs(κ) match to the leading order as κ → −∞. We then analyze a linear programming algorithm to find a solution, and characterize the corresponding threshold δ lin (κ). We observe a gap between the interpolation threshold δs(κ) and the linear programming threshold δ lin (κ), raising the question of the behavior of other algorithms.

show abstract

“…This was argued by proving that the model undergoes the so-called clustering phase transition near α ALG [ART06], [MMZ05] (more on this below), but and also by ruling out various families of algorithms. These algorithms include sequential local algorithms and algorithms based on low-degree polynomials (using the Overlap Gap Property discussed in the next section) [GS17b], [BH21], the Survey Propagation algorithm [Het16], and a variant of a random search algorithm called WalkSAT [COHH17]. The Survey Propagation algorithm is a highly effective heuristics for finding satisfying assignments in random K-SAT and many other similar problems.…”

Section: In Search Of the "Right" Algorithmic Complexity Theorymentioning

confidence: 99%

The Overlap Gap Property: a Geometric Barrier to Optimizing over Random Structures

Gamarnik

2021

Preprint

View full text Add to dashboard Cite

The problem of optimizing over random structures emerges in many areas of science and engineering, ranging from statistical physics to machine learning and artificial intelligence. For many such structures finding optimal solutions by means of fast algorithms is not known and often is believed not possible. At the same time the formal hardness of these problems in form of say complexity-theoretic N P -hardness is lacking.In this introductory article a new approach for algorithmic intractability in random structures is described, which is based on the topological disconnectivity property of the set of pair-wise distances of near optimal solutions, called the Overlap Gap Property. The article demonstrates how this property a) emerges in most models known to exhibit an apparent algorithmic hardness b) is consistent with the hardness/tractability phase transition for many models analyzed to the day, and importantly c) allows to mathematically rigorously rule out large classes of algorithms as potential contenders, in particular the algorithms exhibiting the input stability (insensitivity).

show abstract

The Algorithmic Phase Transition of Random $k$-SAT for Low Degree Polynomials

Cited by 5 publications

References 47 publications

Tight Lipschitz Hardness for Optimizing Mean Field Spin Glasses

Tight Lipschitz Hardness for Optimizing Mean Field Spin Glasses

Tractability from overparametrization: The example of the negative perceptron

The Overlap Gap Property: a Geometric Barrier to Optimizing over Random Structures

Contact Info

Product

Resources

About