Enhancing ENIGMA Given Clause Guidance

Jakubův, Jan; Urban, Josef

doi:10.1007/978-3-319-96812-4_11

Cited by 18 publications

(39 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various possible choices of efficient clause features for theorem prover guidance have been experimented with [14,15,21,22]. The original ENIGMA [14] uses term-tree walks of length 3 as features, while the second version [15] reaches better results by employing various additional features. In particular, the following types of features are used (see [14,Sec Since there are only finitely many features in any training data, the features can be serially numbered.…”

Section: Enigma Clause Featuresmentioning

confidence: 99%

“…When the number of features is greater than the base-which is our case as we intend to use hashing for dimensionality reduction-collisions are inevitable. When using hash base of 32000 (ca 2 15 ) there are almost as many hashing buckets as there are features in the training data (31675). Out of these features, ca 12000 features are hashed without a collision and 12000 buckets are unoccupied.…”

Section: Evaluation Of Feature Hashingmentioning

confidence: 99%

“…In this work, we add two state-of-the-art machine learning methods to the ENIGMA [14,15] algorithm that efficiently guides saturation-style proof search. The first one trains gradient boosted trees on efficiently extracted manually designed (handcrafted) clause features.…”

mentioning

confidence: 99%

See 2 more Smart Citations

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

Chvalovský

Jakubův

Suda

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We describe an efficient implementation of clause guidance in saturation-based automated theorem provers extending the ENIGMA approach. Unlike in the first ENIGMA implementation where fast linear classifier is trained and used together with manually engineered features, we have started to experiment with more sophisticated state-of-the-art machine learning methods such as gradient boosted trees and recursive neural networks. In particular the latter approach poses challenges in terms of efficiency of clause evaluation, however, we show that deep integration of the neural evaluation with the ATP data-structures can largely amortize this cost and lead to competitive real-time results. Both methods are evaluated on a large dataset of theorem proving problems and compared with the previous approaches. The resulting methods improve on the manually designed clause guidance, providing the first practically convincing application of gradient-boosted and neural clause guidance in saturation-style automated theorem provers. IntroductionAutomated theorem provers (ATPs) [32] have been developed for decades by manually designing proof calculi and search heuristics. Their power has been growing and they are already very useful, e.g., as parts of large interactive theorem proving (ITP) verification toolchains (hammers) [5]. On the other hand, with small exceptions, ATPs are still significantly weaker than trained mathematicians in finding proofs in most research domains.Recently, machine learning over large formal corpora created from ITP libraries [37,28,19] has started to be used to develop guidance of ATP systems [39,25,2]. This has already produced strong systems for selecting relevant facts for proving new conjectures over large formal libraries [1,4,9]. More recently, machine learning has also started to be used to guide the internal search of the ATP systems. In sophisticated saturation-style provers this has been done by feedback loops for strategy invention [38,16,33] and by using supervised learning [14,26] to select the next given clause [27]. In the simpler connection tableau systems such as leancop [29], supervised learning has been used to choose ⋆ Supported by the ERC Consolidator grant no. 649043 AI4REASON, and by the Czech project AI&Reasoning CZ.02.1.01/0.0/0.0/15 003/0000466 and the European Regional Development Fund.

show abstract

Section: Enigma Clause Featuresmentioning

confidence: 99%

Section: Evaluation Of Feature Hashingmentioning

confidence: 99%

See 1 more Smart Citation

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

Chvalovský

Jakubův

Suda

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Various machine learning methods can handle numeric vectors and their success heavily depends on the selection of correct clause features. Various possible choices of efficient clause features for theorem prover guidance have been experimented with [6,7,10,11]. The original ENIGMA [6] uses term-tree walks of length 3 as features, while the second version [7] reaches better results by employing various additional features.…”

Section: Enigma: Learning From Successful Proof Searchesmentioning

confidence: 99%

ENIGMAWatch: ProofWatch Meets ENIGMA

Goertzel

Jakubův

Urban

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

In this work we describe a new learning-based proof guidance -ENIGMAWatch -for saturation-style first-order theorem provers. ENIGMAWatch combines two guiding approaches for the given-clause selection implemented for the E ATP system: ProofWatch and ENIGMA. ProofWatch is motivated by the watchlist (hints) method and based on symbolic matching of multiple related proofs, while ENIGMA is based on statistical machine learning. The two methods are combined by using the evolving information about symbolic proof matching as an additional information that characterizes the saturation-style proof search for the statistical learning methods. The new system is experimentally evaluated on a large set of problems from the Mizar library. We show that the added proof-matching information is considered important by the statistical machine learners, and that it leads to improvements in E's performance over ProofWatch and ENIGMA.

show abstract

“…This work proposes and develops a new learning-based proof guidance -ENIGMAWatch -for saturation-style first-order theorem provers. It is based on two previous guiding methods implemented for the E [13] ATP system: ProofWatch [4] and ENIGMA [7,8]. Both ProofWatch and ENIGMA enable E to use related proofs for guiding the proof search for a new conjecture.…”

Section: Introductionmentioning

confidence: 99%

ProofWatch Meets ENIGMA: First Experiments

Goertzel¹,

Jakubův²,

Urban³

Kalpa Publications in Computing

Self Cite

View full text Add to dashboard Cite

Watchlist (also hint list) is a technique that allows lemmas from related proofs to guide a saturation-style proof search for a new conjecture. ProofWatch is a recent watchlist-style method that loads many previous proofs inside the ATP, maintains their completion ratios during the proof search and focuses the search by following the most completed proofs. In this work, we start to use the dynamically changing vector of proof completion ratios as additional information about the saturation-style proof state for statistical machine learning methods that evaluate the generated clauses. In particular, we add the proof completion vectors to ENIGMA (efficient learning-based inference guiding machine) and evaluate the new method on the MPTP Challenge benchmark, showing moderate improvement in E’s performance over ProofWatch and ENIGMA.

show abstract

Enhancing ENIGMA Given Clause Guidance

Cited by 18 publications

References 7 publications

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

ENIGMAWatch: ProofWatch Meets ENIGMA

ProofWatch Meets ENIGMA: First Experiments

Contact Info

Product

Resources

About