The overlap gap property: A topological barrier to optimizing over random structures

Abstract

The problem of optimizing over random structures emerges in many areas of science and engineering, ranging from statistical physics to machine learning and artificial intelligence. For many such structures, finding optimal solutions by means of fast algorithms is not known and often is believed not to be possible. At the same time, the formal hardness of these problems in the form of the complexity-theoretic NP-hardness is lacking. A new approach for algorithmic intractability in random structures is described in this article, which is based on the topological disconnectivity property of the set of pairwise distances of near-optimal solutions, called the Overlap Gap Property. The article demonstrates how this property 1) emerges in most models known to exhibit an apparent algorithmic hardness; 2) is consistent with the hardness/tractability phase transition for many models analyzed to the day; and, importantly, 3) allows to mathematically rigorously rule out a large class of algorithms as potential contenders, specifically the algorithms that exhibit the input stability (insensitivity).

Nonequilibrium Monte Carlo for unfreezing variables in hard combinatorial optimization

“…Overall, we believe that our algorithm could have wide range of applications for combinatorial problems, mostly as a subroutine in conjunction with existing high performant solvers, by essentially reducing the cardinality of the subset of worst-case instances, or reducing the algorithmic gap [75]. There are also significant challenges for learning and inference in structured graphical models with well-known computational bottlenecks related to the hardness of evaluation of marginal probability distributions or evaluation of partition functions.…”

“…For random 4-SAT near the computational phase transition, finding frozen cores of size O(N ) via the whitening procedure indicates the existence of rarely observed low-energy solutions residing beyond energy barriers that are widely believed to be exponentially hard to penetrate [17]. This is related to the concept of "overlap gap property ", or topological barrier in solution space of random structures, that has been recently used to explain algorithmic gaps, or absence of polynomial performance for a large class of algorithms in a regime between condensation phase transition and the actual computational phase transition [75].…”

Maximum Consensus by Weighted Influences of Monotone Boolean Functions

“…Here, we introduce a new Boolean cube {0, 1} n0 , for any (15) have the following order relationship ∀ i i i, j j j ∈ {0, 1} n0 , i i i j j j =⇒ f q (S i i i ) < f q (S j j j ). (11) Note that f q (S • ) exists only if S • is non-empty.…”

“…This leads us to conjecture: Conjecture 3.1. (Complexity of MaxCon and Overlap Gap Property (OGP)) In [11] (and the works referred therein), the computational complexity of many algorithms has been linked to what is called the OGP. Essentially it is argued that if the solution space is highly clustered (and here locality is measured by overlap of the solutions) then the problem will be hard for whole classes of algorithms.…”

“…This conjecture is particularly intriguing because of the link between maximum independent set (studied in [11]) and MaxCon -it can be shown that the MaxCon solution is the maximum independent set of the (hyper)graph formed by the infeasible minimal sized subsets (and also to the minimum vertex cover of the complement hypergraph).…”