ENIGMA: Efficient Learning-Based Inference Guiding Machine

Abstract

ENIGMA is a learning-based method for guiding given clause selection in saturationbased theorem provers. Clauses from many proof searches are classified as positive and negative based on their participation in the proofs. An efficient classification model is trained on this data, using fast feature-based characterization of the clauses . The learned model is then tightly linked with the core prover and used as a basis of a new parameterized evaluation heuristic that provides fast ranking of all generated clauses. The approach is evaluated on the E prover and the CASC 2016 AIM benchmark, showing a large increase of E's performance. Introduction: Theorem Proving and LearningState-of-the-art resolution/superposition automated theorem provers (ATPs) such as Vampire [15] and E [20] are today's most advanced tools for general reasoning across a variety of mathematical and scientific domains. The stronger the performance of such tools, the more realistic become tasks such as full computer understanding and automated development of complicated mathematical theories, and verification of software, hardware and engineering designs. While performance of ATPs has steadily grown over the past years due to a number of human-designed improvements, it is still on average far behind the performance of trained mathematicians. Their advanced knowledge-based proof finding is an enigma, which is unlikely to be deciphered and programmed completely manually in near future.On large corpora such as Flyspeck, Mizar and Isabelle, the ATP progress has been mainly due to learning how to select the most relevant knowledge, based on many previous proofs [10,12,1,2]. Learning from many proofs has also recently become a very useful method for automated finding of parameters of ATP strategies [22,9,19,16], and for learning of sequences of tactics in interactive theorem provers (ITPs) [7]. Several experiments with the compact leanCoP [18] system have recently shown that directly using trained machine learner for internal clause selection can significantly prune the search space and solve additional problems [24,11,5]. An obvious next step is to implement efficient learning-based clause selection also inside the strongest superposition-based ATPs.In this work, we introduce ENIGMA -Efficient learNing-based Internal Guidance MAchine for state-of-the-art saturation-based ATPs. The method applies fast machine learning algorithms to a large number of proofs, and uses the trained classifier together with simpler heuristics to evaluate the millions of clauses generated during the resolution/superposition proof search. This way, the theorem prover automatically takes into account thousands of previous successes and failures that it has seen in previous problems, similarly to trained humans. Thanks to a carefully chosen efficient learning/evaluation method and its tight integration with the core ATP (in our case the E prover), the penalty for this ubiquitous knowledge-based internal proof guidance is very low. This in turn very significantly improves the per...

ENIGMAWatch: ProofWatch Meets ENIGMA

“…Various machine learning methods can handle numeric vectors and their success heavily depends on the selection of correct clause features. Various possible choices of efficient clause features for theorem prover guidance have been experimented with [6,7,10,11]. The original ENIGMA [6] uses term-tree walks of length 3 as features, while the second version [7] reaches better results by employing various additional features.…”

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

“…More recently, machine learning has also started to be used to guide the internal search of the ATP systems. In sophisticated saturation-style provers this has been done by feedback loops for strategy invention [38,16,33] and by using supervised learning [14,26] to select the next given clause [27]. In the simpler connection tableau systems such as leancop [29], supervised learning has been used to choose ⋆ Supported by the ERC Consolidator grant no.…”

“…In this work, we add two state-of-the-art machine learning methods to the ENIGMA [14,15] algorithm that efficiently guides saturation-style proof search. The first one trains gradient boosted trees on efficiently extracted manually designed (handcrafted) clause features.…”

AI at CADE/IJCAR

“…I am not going to argue and explain more here -I have written enough on this topic in the past fifteen years [5,6,7,9,11,10,2,8,3,1]. Despite Dmitriy's and Giles's honest efforts, it is hard to believe that it would not be a wasted effort, as were my occasional replies to the most appalling reviews in the past.…”