ProofWatch Meets ENIGMA: First Experiments

Goertzel, Zarathustra; Jakubův, Jan; Urban, Josef

doi:10.29007/z7qx

Cited by 4 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are a number of future directions. For example, research in better proof state characterization of saturation-style systems has been started recently [10,11] and it is likely that evolving vectorial representations of the proof state will further contribute to the quality of the learning-based guidance. Our recursive neural model is just one of many, and a number of related and combined models can be experimented with.…”

Section: Discussionmentioning

confidence: 99%

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

Chvalovský

Jakubův

Suda

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We describe an efficient implementation of clause guidance in saturation-based automated theorem provers extending the ENIGMA approach. Unlike in the first ENIGMA implementation where fast linear classifier is trained and used together with manually engineered features, we have started to experiment with more sophisticated state-of-the-art machine learning methods such as gradient boosted trees and recursive neural networks. In particular the latter approach poses challenges in terms of efficiency of clause evaluation, however, we show that deep integration of the neural evaluation with the ATP data-structures can largely amortize this cost and lead to competitive real-time results. Both methods are evaluated on a large dataset of theorem proving problems and compared with the previous approaches. The resulting methods improve on the manually designed clause guidance, providing the first practically convincing application of gradient-boosted and neural clause guidance in saturation-style automated theorem provers. IntroductionAutomated theorem provers (ATPs) [32] have been developed for decades by manually designing proof calculi and search heuristics. Their power has been growing and they are already very useful, e.g., as parts of large interactive theorem proving (ITP) verification toolchains (hammers) [5]. On the other hand, with small exceptions, ATPs are still significantly weaker than trained mathematicians in finding proofs in most research domains.Recently, machine learning over large formal corpora created from ITP libraries [37,28,19] has started to be used to develop guidance of ATP systems [39,25,2]. This has already produced strong systems for selecting relevant facts for proving new conjectures over large formal libraries [1,4,9]. More recently, machine learning has also started to be used to guide the internal search of the ATP systems. In sophisticated saturation-style provers this has been done by feedback loops for strategy invention [38,16,33] and by using supervised learning [14,26] to select the next given clause [27]. In the simpler connection tableau systems such as leancop [29], supervised learning has been used to choose ⋆ Supported by the ERC Consolidator grant no. 649043 AI4REASON, and by the Czech project AI&Reasoning CZ.02.1.01/0.0/0.0/15 003/0000466 and the European Regional Development Fund.

show abstract

Section: Discussionmentioning

confidence: 99%

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

Chvalovský

Jakubův

Suda

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…We have produced and evaluated the first practically usable version the ENIG-MAWatch system which can now be efficiently used over large mathematical datasets. The previous experiments with the first prototype on the small MPTP Challenge [4] demonstrated that ENIGMAWatch can find proofs faster (in terms of how many processed clauses are needed). The work presented here shows that with improved subsumption indexing, feature hashing, and suitable global watchlist selection, ENIGMAWatch outperforms ENIGMA on the large Mizar40 dataset.…”

Section: Discussionmentioning

confidence: 99%

“…In E, the watchlist mechanism is implemented using a priority function 3 which takes precedence over weight function used to select the next given clause. Priority functions assign the priority to each clause, and clauses with higher priority are selected as given before clauses with lower priority 4 . When clauses from previous proofs are put on a watchlist, E thus prefers to follow steps from the previous proofs whenever it can.…”

Section: Proofwatch: Proof Guidance By Clause Subsumptionmentioning

confidence: 99%

See 1 more Smart Citation

ENIGMAWatch: ProofWatch Meets ENIGMA

Goertzel

Jakubův

Urban

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

In this work we describe a new learning-based proof guidance -ENIGMAWatch -for saturation-style first-order theorem provers. ENIGMAWatch combines two guiding approaches for the given-clause selection implemented for the E ATP system: ProofWatch and ENIGMA. ProofWatch is motivated by the watchlist (hints) method and based on symbolic matching of multiple related proofs, while ENIGMA is based on statistical machine learning. The two methods are combined by using the evolving information about symbolic proof matching as an additional information that characterizes the saturation-style proof search for the statistical learning methods. The new system is experimentally evaluated on a large set of problems from the Mizar library. We show that the added proof-matching information is considered important by the statistical machine learners, and that it leads to improvements in E's performance over ProofWatch and ENIGMA.

show abstract

“…While developing this kind of efficient internal guidance for state-of-the-art saturation ATPs has been challenging and took time, the very large gains obtained here show that this has been very well invested effort. Future work will certainly focus on even stronger learning methods and also on more dynamic proof state characterization such as ProofWatch [12] and ENIGMAWatch [13]. It is however clear that this is the point when machine learning guidance has very strongly overtaken the human development of ATP strategies over large problem corpora.…”

Section: Discussionmentioning

confidence: 99%

Hammering Mizar by Learning Clause Guidance

Jakubův,

Urban

2019

Preprint

Self Cite

View full text Add to dashboard Cite

We describe a very large improvement of existing hammer-style proof automation over large ITP libraries by combining learning and theorem proving. In particular, we have integrated state-ofthe-art machine learners into the E automated theorem prover, and developed methods that allow learning and efficient internal guidance of E over the whole Mizar library. The resulting trained system improves the real-time performance of E on the Mizar library by 70% in a single-strategy setting.

show abstract

ProofWatch Meets ENIGMA: First Experiments

Cited by 4 publications

References 12 publications

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

ENIGMA-NG: Efficient Neural and Gradient-Boosted Inference Guidance for E

ENIGMAWatch: ProofWatch Meets ENIGMA

Hammering Mizar by Learning Clause Guidance

Contact Info

Product

Resources

About