Neural Circuit Synthesis from Specification Patterns

Abstract

We train hierarchical Transformers on the task of synthesizing hardware circuits directly out of high-level logical specifications in linear-time temporal logic (LTL). The LTL synthesis problem is a well-known algorithmic challenge with a long history and an annual competition is organized to track the improvement of algorithms and tooling over time. New approaches using machine learning might open a lot of possibilities in this area, but suffer from the lack of sufficient amounts of training data. In this paper, we consider a method to generate large amounts of additional training data, i.e., pairs of specifications and circuits implementing them. We ensure that this synthetic data is sufficiently close to human-written specifications by mining common patterns from the specifications used in the synthesis competitions. We show that hierarchical Transformers trained on this synthetic data solve a significant portion of problems from the synthesis competitions, and even out-of-distribution examples from a recent case study.Preprint. Under review.

Generating Symbolic Reasoning Problems with Transformer GANs

“…A simplified NeuroSAT architecture was trained for unsat-core predictions (Selsam & Bjørner, 2019). Neural networks have been applied to 2QBF (Lederman et al, 2020), logical entailment (Evans et al, 2018), SMT (Balunovic et al, 2018), and temporal logics (Hahn et al, 2020;Schmitt et al, 2021).…”

“…Deep learning is on the verge of transitioning from traditional application domains, like image recognition (He et al, 2015), face recognition (Taigman et al, 2014), or translation (Wu et al, 2016), to domains that involve complex symbolic reasoning tasks. Examples include the application of deep neural network architectures to SAT (Selsam et al, 2019;Selsam & Bjørner, 2019;Ozolins et al, 2021), SMT (Balunovic et al, 2018), temporal specifications in verification (Hahn et al, 2020;Schmitt et al, 2021), symbolic mathematics (Lample & Charton, 2020), or theorem proving (Loos et al, 2017;Bansal et al, 2019;Huang et al, 2019;Urban & Jakubuv, 2020).…”

“…Since the instances are too few to be trained on, training data is, thus, typically generated synthetically. For example by random sampling (Selsam et al, 2019;Lample & Charton, 2020), or by randomly re-combining parts of existing instances (Schmitt et al, 2021). Although these data generation methods already lead to good results, training on randomly generated data carries the risk of training on meaningless data or the risk of introducing unwanted biases.…”