Predicting Propositional Satisfiability via End-to-End Learning

Abstract

Strangely enough, it is possible to use machine learning models to predict the satisfiability status of hard SAT problems with accuracy considerably higher than random guessing. Existing methods have relied on extensive, manual feature engineering and computationally complex features (e.g., based on linear programming relaxations). We show for the first time that even better performance can be achieved by end-to-end learning methods — i.e., models that map directly from raw problem inputs to predictions and take only linear time to evaluate. Our work leverages deep network models which capture a key invariance exhibited by SAT problems: satisfiability status is unaffected by reordering variables and clauses. We showed that end-to-end learning with deep networks can outperform previous work on random 3-SAT problems at the solubility phase transition, where: (1) exactly 50% of problems are satisfiable; and (2) empirical runtimes of known solution methods scale exponentially with problem size (e.g., we achieved 84% prediction accuracy on 600-variable problems, which take hours to solve with state-of-the-art methods). We also showed that deep networks can generalize across problem sizes (e.g., a network trained only on 100-variable problems, which typically take about 10 ms to solve, achieved 81% accuracy on 600-variable problems).

Machine Learning for Automated Theorem Proving: Learning to Solve SAT and QSAT

“…Another approach, related to the GNN methods described above, involves learning features from experience without designing them by hand; an example is the work of Loreggia et al (2016) that we describe in Section 6.5. There is some evidence (Cameron et al, 2020) that such features can be faster to compute than engineered alternatives. We shall discuss these issues further as they arise in later Chapters.…”

DOGE-Train: Discrete Optimization on GPU with End-to-end Training

“…To our knowledge it is the only work that addresses computing ILP relaxations with ML. For constraint satisfaction problems [40,9,47] train GNN while [47] train in an unsupervised manner. For narrow subclasses of problems primal heuristics have been augmented through learning some of their decisions, e.g.…”

An overview of machine learning techniques in constraint solving

“…With the focus of predicting the satisfiability of individual 3-SAT problems (maximum three literals per clause), Cameron et al (2020) introduce a deep neural network based approach based on the concept of end-to-end learning (all model parameters are learned at the same time). In this context, the authors introduce two different network architectures, one focusing on the prediction of satisfiability, the other one supporting the prediction of satisfiable variable assignments.…”