Hard optimization problems have soft edges

Abstract

Finding a Maximum Clique is a classic property test from graph theory; find any one of the largest complete subgraphs in an Erdös-Rényi G(N, p) random graph. We use Maximum Clique to explore the structure of the problem as a function of N, the graph size, and K, the clique size sought. It displays a complex phase boundary, a staircase of steps at each of which $$2 \log _2N$$ 2 log 2 N and $$K_{\text {max}}$$ K max , the maximum size of a clique that can be found, increases by 1. Each of its boundaries has a finite width, and these widths allow local algorithms to find cliques beyond the limits defined by the study of infinite systems. We explore the performance of a number of extensions of traditional fast local algorithms, and find that much of the “hard” space remains accessible at finite N. The “hidden clique” problem embeds a clique somewhat larger than those which occur naturally in a G(N, p) random graph. Since such a clique is unique, we find that local searches which stop early, once evidence for the hidden clique is found, may outperform the best message passing or spectral algorithms.

Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems

“…The effectiveness of MC methods is witnessed by their success in hard discrete combinatorial problems [7][8][9][10][11][12] . The theory beyond MC algorithms is very strong thanks to the theory of Markov chains and many concepts borrowed from the principles of statistical mechanics (e.g.…”

Phase transitions in the mini-batch size for sparse and dense two-layer neural networks

“…perceptron, one hidden layer, committee machine [14][15][16]) are well known and understood, when the algorithmic dynamics is governed by equilibrium processes [17]. However, the out-of-equilibrium dynamics of the learning processes [18][19][20] are much more difficult to study and most of the results about it are restricted to dense models where the Martin-Siggia-Rose formalism [21] can be applied to derive DMFT equations [22,23].…”

Large Independent Sets on Random d-Regular Graphs with Fixed Degree d

“…In addition to having several direct applications [9], the MIS is closely related to another well-known optimization problem, the maximum clique problem [10,11]. In order to find the maximum clique (the largest complete subgraph) of a graph G( Ñ , E), it suffices to search for the maximum independent set of the complement of G( Ñ , E).…”