The large deviations of the whitening process in random constraint satisfaction problems

Abstract

V. Numerical experiments 32 VI. Conclusions and perspectives 34 Acknowledgments 35 A. Technical details on the cavity treatment 35 1. Simplifying the Belief Propagation equations 35 2. Finite time horizon 38 3. Explicit time clipping 40 4. The regular graph case 40B. The large k limit of the T = 1 results 42 1. The large k limit of the entropy function 42 2. The tipping point 43 3. The asymptotic behavior of the threshold l 1 (k) 44C. The fixed points of the whitening (large T limit) 44 1. A more compact equivalent form of the RS equations 44 2. A dynamical system point of view 45 3. The unfrozen solutions for l > l r 45 4. The frozen solutions of the first kind 47 5. The frozen solutions of the second kind 48 6. The large k expansion of the entropy s ∞ (θ = 0) and of the threshold l ∞ (k) 49 D. The large k limit of the threshold l T (k) for 1 < T < ∞ 50References 52 SAT instances), but we believe our conclusions are qualitatively valid for a wide family of random CSP, and even quantitatively in the large k limit. We obtained results for finite values of k, and also an asymptotic expansion of α f at large k which, once translated in terms of the k-SAT problem, reads at the leading order α f ∼ 2 k−1 ln 2, i.e. on the scale α sat /2, much larger than α r ∼ α sat /k (forgetting the subdominant logarithmic correction): atypical unfrozen solutions survive the addition of a huge amount of constraints besides the typical rigidity threshold. Note that this asymptotic scale α sat /2 appeared in [52] as the density above which the range of possible Hamming distances between pairs of solutions becomes gapped. The location of this freezing transition was not computed previously by statistical physics calculations. It has been addressed numerically for the random 3-SAT problem [53] where it was shown that for this case the frozen region is tiny, taking only about 0.3% of the satisfiable phase. Two rigorous works placed an upper bound on the freezing transition. Particularly for random k-SAT is it known that for large k all clusters are frozen when α > (4/5)α sat [26], and for random coloring there also exists a proof that there is a gap between the freezing and the satisfiability transition [54]. From a technical point of view our work amounts to a characterization of the large deviation properties of the whitening dynamics introduced above (building on recent studies of rare events in the related bootstrap percolation dynamics [55][56][57]). To this end we introduced a biased measure over the set of solutions, the bias being a function of the number of frozen variables after T steps of (a parallel version of) the whitening dynamics, that induces interactions between variables at distances up to T in the factor graph. The determination of α f requires to take the limit T → ∞, in order to characterize the fixed points of the whitening, which we managed to do analytically. In addition we also studied the structure of the biased measure over solutions for finite values of T , and discovered the following phenomenon. For each val...

Phase transitions in theq-coloring of random hypergraphs

“…an expansion valid when q and/or K diverges (see Appendix D for details). The constant term in the parenthesis depends on the Parisi parameter m, we computed its value in the two special cases m = 1 and m = 0, finding C(m = 1) = 1 and C(m = 0) = 1−ln 2 (in agreement with previous results for K = 2 [17,18] and q = 2 [27]). We also expect the clustering transition clust to have the same asymptotic expansion as in (67), with a different constant term C; this has been rigorously proven for coloring of graphs (K = 2) in [19,23].…”

“…The colorability and condensation thresholds have been found in previous studies [7,17,18,27,38] to occur asymptotically very close to the upperbound of the first moment method. This is also the case here, we have indeed obtained:…”

“…The conclusion of this analysis is that clust = cond = RS stab in the case q = 2, K = 4, and that the coexistence of solutions of the 1RSB equations at m = 1 is not thermodynamically relevant here: the minimum in (44) is reached at a non-trivial value m s < 1, hence neither of the solutions at m = 1 yields the correct 1RSB prediction for the entropy. Note that this peculiar bifurcation scenario remained, as far as we know, unobserved previously, and that it caused a slight mistake in [27].…”

Monte Carlo algorithms are very effective in finding the largest independent set in sparse random graphs

“…For this reason, the freezing threshold is conjectured to be the ultimate algorithmic threshold. Unfortunately, its analytic computation is a very difficult task, which has been achieved only in random hypergraph bi-coloring at present [14].…”

Role of Synaptic Stochasticity in Training Low-Precision Neural Networks

“…This apparent inconsistency has been solved through a large deviation analysis which revealed the existence of sub-dominant and dense regions of solutions [14,23]. This analysis introduced the concept of Local Entropy [14] which subsequently led to other algorithmic developments [24][25][26] (see also [27] for related analysis). In the generalization perspective, solutions within a dense region may be loosely considered as representative of the entire region itself, and therefore act as better pointwise predictors than isolated solutions, since the optimal Bayesian predictor is obtained averaging all solutions [14].…”