Strongly refuting random CSPs below the spectral threshold

Abstract

Random constraint satisfaction problems (CSPs) are known to exhibit threshold phenomena: given a uniformly random instance of a CSP with n variables and m clauses, there is a value of m = Ω(n) beyond which the CSP will be unsatisfiable with high probability. Strong refutation is the problem of certifying that no variable assignment satisfies more than a constant fraction of clauses; this is the natural algorithmic problem in the unsatisfiable regime (when m/n = ω(1)).Intuitively, strong refutation should become easier as the clause density m/n grows, because the contradictions introduced by the random clauses become more locally apparent. For CSPs such as k-SAT and k-XOR, there is a long-standing gap between the clause density at which efficient strong refutation algorithms are known, m/n ≥ O(n k/2−1 ), and the clause density at which instances become unsatisfiable with high probability, m/n = ω(1).In this paper, we give spectral and sum-of-squares algorithms for strongly refuting random k-XOR instances with clause density m/n ≥ O(n (k/2−1)(1−δ) ) in time exp( O(n δ )) or in O(n δ ) rounds of the sum-of-squares hierarchy, for any δ ∈ [0, 1) and any integer k ≥ 3. Our algorithms provide a smooth transition between the clause density at which polynomial-time algorithms are known at δ = 0, and brute-force refutation at the satisfiability threshold when δ = 1. We also leverage our k-XOR results to obtain strong refutation algorithms for SAT (or any other Boolean CSP) at similar clause densities.Our algorithms match the known sum-of-squares lower bounds due to Grigoriev and Schonebeck, up to logarithmic factors.First we will bound the value of term in (3.9). Recall that by Lemma 3.7, if E is even then the number of distinct labels in V ∈ V is less than E 0 . Therefore,Now, we will use the following claim:Proof. If H σ is diagonal-free and even, then we claim that each pivot value appears twice. Suppose not, if σ i is such that σ i = σ j for all j = i. Since H σ is diagonal free, the two hyperedges involving σ i are distinct. Since this is the unique occurrence of these two hyperedges in H σ , H σ cannot be even-a contradiction. With each pivot appearing at least twice, the number of distinct choices of σ is at most (2d )!n d .

The Power of Sum-of-Squares for Detecting Hidden Structures

“…Initiated by the work of [BBH + 12], who showed that polynomial time algorithms in the hierarchy solve all known integrality gap instances for Unique Games and related problems, a steady stream of works have developed efficient algorithms for both worst-case [BKS14, BKS15, BKS17, BGG + 16] and average-case problems [HSS15,GM15,BM16,RRS16,BGL16,MSS16a,PS17]. The insights from these works extend beyond individual algorithms to characterizations of broad classes of algorithmic techniques.…”

“…Key examples are finding solutions of random constraint satisfaction problems (CSPs) with planted assignments [RRS16] and finding planted optima of random polynomials over the n-dimensional unit sphere [RRS16,BGL16]. The latter formulation captures a wide range of unsupervised learning problems, and has led to many unsupervised learning algorithms with the best-known polynomial time guarantees [BKS15, BKS14, MSS16b, HSS15, PS17, BGG + 16].…”

“…The twist is that the entries of these matrices are low-degree polynomials in the input to the algorithm . The result is a new family of low-degree spectral algorithms with guarantees matching SoS but requriring only eigenvector computations instead of general semidefinite programming [HSSS16,RRS16,AOW15a].…”

“…For both problems there are nontrivial low-degree spectral algorithms, which have better noise tolerance than naive spectral methods [HSSS16,DM14b,RRS16,BGL16]. Sparse PCA, which is used in machine learning and statistics to find important coordinates in high-dimensional data sets, has attracted much attention in recent years for being apparently computationally intractable to solve with a number of samples which is more than sufficient for brute-force algorithms [KNV + 15, BR13b,MW15a].…”

Extended Formulation Lower Bounds for Refuting Random CSPs

“…This leaves open a wide range of densities from constant to n k−2 2 where no polynomial time algorithms for refutation are known. It was recently shown that degreeÕ(n ε ) sum-of-squares semidefinite programs can refute random k-SAT instances at density n ( k−2 2 )(1−ε) for all ε ∈ (0, 1), thereby yielding a subexponential time refutation algorithm for all densities in this range [RRS16]. However, the problem By using the fact that the k-XOR predicate and the k-SAT predicate support (k −1)-wise uniform distribution on satisfying assignments, the following corollary is immediate.…”