Universal Policies for Software-Defined MDPs

Selsam, Daniel; Han, Jesse Michael; Moura, Leonardo de; Godefroid, Patrice

doi:10.48550/arxiv.2012.11401

Cited by 1 publication

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We train a teacher agent and a solver agent in sequence using the (Gumbel) AlphaZero algorithm. Both agents use a similar 1.6M parameters Dynamic Graph Transformer network [30] (see architecture details in Appendix H). The teacher agent is trained first for 20 AlphaZero iterations.…”

Section: Training Protocolmentioning

confidence: 99%

“…One proposal [30] is to use supervised learning for training a universal oracle that is conditioned on the description of arbitrary search problems and can therefore use training data from a large number of heterogeneous sources. In contrast, we propose using reinforcement learning with the insight that nondeterministic search strategies can be used to generate problems in addition to solving them.…”

Section: Related Workmentioning

confidence: 99%

“…Our proposed network architecture works as follows. Given a choice point, the probe is encoded using a Dynamic Graph Transformer (DGT) neural network [30]. DGT networks are similar to Transformers [40] but they allow leveraging some additional graph structure over the source tokens by associating edge types to learned attention biases.…”

Section: G Training Hyperparametersmentioning

confidence: 99%

See 2 more Smart Citations

Learning to Find Proofs and Theorems by Learning to Refine Search Strategies: The Case of Loop Invariant Synthesis

Laurent¹,

Platzer²

2022

Preprint

View full text Add to dashboard Cite

We propose a new approach to automated theorem proving and deductive program synthesis where an AlphaZero-style agent is self-training to refine a high-level expert strategy expressed as a nondeterministic program. An analogous teacher agent is self-training to generate tasks of suitable relevance and difficulty for the learner. This allows leveraging minimal amounts of domain knowledge to tackle problems for which training data is unavailable or hard to synthesize. We illustrate our approach on the problem of loop invariant synthesis for imperative programs and using neural networks to refine both the teacher and solver strategies.Preprint. Under review.

show abstract

Section: Training Protocolmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%