HyperTree Proof Search for Neural Theorem Proving

Abstract

We propose an online training procedure for a transformer-based automated theorem prover. Our approach leverages a new search algorithm, HyperTree Proof Search (HTPS), inspired by the recent success of AlphaZero. Our model learns from previous proof searches through online training, allowing it to generalize to domains far from the training distribution. We report detailed ablations of our pipeline's main components by studying performance on three environments of increasing complexity. In particular, we show that with HTPS alone, a model trained on annotated proofs manages to prove 65.4% of a held-out set of Metamath theorems, significantly outperforming the previous state of the art of 56.5% by GPT-f. Online training on these unproved theorems increases accuracy to 82.6%. With a similar computational budget, we improve the state of the art on the Lean-based miniF2F-curriculum dataset from 31% to 42% proving accuracy.

LEMMA: Bootstrapping High-Level Mathematical Reasoning with Learned Symbolic Abstractions

“…Our work proposes to augment methods that alternate between learning and search to solve mathematical reasoning problems. Several methods of this flavor have been introduced recently, such as Expert Iteration [1] and AlphaZero [13] for game playing, HTPS [6] and GPT-f [11] for neural theorem proving, and ConPoLe [10] for mathematical problems from the Common Core environments.…”

“…The Common Core environments are a simplified setting compared to industrial theorem proving languages, like Lean [8] or Isabelle/HOL [9]. A range of significant work has been developed in RL agents to find proofs in these languages [11,6,3], typically using a language model fine-tuned on human proofs as the action generator.…”

“…Systematic search for sequences of steps can dramatically benefit from learning: patterns from previous problems can help guide future searches towards more promising sequences. This idea of combining search and learning has been widely applied, with notable domains including neural theorem proving [3,5,6,11,16], program synthesis [2,4,15] and game playing [1,13].…”