On Repair with Probabilistic Attribute Grammars

Abstract

Program synthesis and repair have emerged as an exciting area of research, driven by the potential for revolutionary advances in programmer productivity. Among most promising ideas emerging for synthesis are syntax-driven search, probabilistic models of code, and the use of input-output examples. Our paper shows how to combine these techniques and use them for program repair, which is among the most relevant applications of synthesis to general-purpose code. Our approach combines semantic specifications, in the form of pre-and post-conditions and input-output examples with syntactic specifications in the form of term grammars and AST-level statistics extracted from code corpora. We show that synthesis in this framework can be viewed as an instance of graph search, permitting the use of well-understood families of techniques such as A*. We implement our algorithm in a framework for verification, synthesis and repair of functional programs, demonstrating that our approach can repair programs that are beyond the reach of previous tools.

Just-in-time learning for bottom-up enumerative synthesis

“…Despite their simplicity, enumerative program synthesizers are known to be very effective: ESolver and EUSolver [Alur et al 2017b] have been past winners of the SyGuS competition [Alur et al 2016[Alur et al , 2017a. Enumerative synthesizers typically explore the space of programs either top-down, by extending a partial program tree from the node towards the leaves [Alur et al 2017b;Kalyan et al 2018;Koukoutos et al 2017;Lee et al 2018], or bottom-up, by gradually building up a program tree from the leaves towards the root [Albarghouthi et al 2013;Udupa et al 2013]. These two strategies have complementary strengths and weaknesses, similar to backward chaining and forward chaining in proof search.…”

“…The general idea is to prioritize the exploration of grammar productions based on scores assigned by a probabilistic model; the specific technique, however, varies depending on (1) the context taken into consideration by the model when assigning scores, and (2) how the scores are taken into account during search. Like Probe, [Balog et al 2016;Koukoutos et al 2017;Menon et al 2013] use a PCFG, which assigns scores to productions independently of their context within the synthesized program; unlike Probe, however, these techniques select the PCFG once, at the beginning of the synthesis process, based on a learned mapping from semantic specifications to scores. On the opposite end of the spectrum, Metal [Si et al 2019] and Concord [Chen et al 2020] use graph-based and sequence-based models, respectively, to condition the scores on the entire partial program that is being extended.…”

“…The more context a model takes into account, the more precise the guidance it provides, but also the harder it is to learn. Another consideration is that neural models, used in [Chen et al 2020;Kalyan et al 2018;Si et al 2019] incur a larger overhead than simple grammar-based models, used in Probe and [Balog et al 2016;Koukoutos et al 2017;Lee et al 2018;Menon et al 2013], since they have to invoke a neural network at each branching point during search.…”

“…As for using the scores to guide search, most existing techniques are specific to top-down enumeration. They include prioritized depth-first search [Balog et al 2016], branch and bound search [Kalyan et al 2018], and variants of best-first search [Koukoutos et al 2017;Lee et al 2018;Menon et al 2013]. In contrast to these approaches, Probe uses the scores to guide bottom-up enumeration with observational equivalence reduction.…”

“…During top-down enumeration, [Koukoutos et al 2017] employs an optimization strategy where the cost of an incomplete (partial) program is lowered if it satisfies some of the examples. This optimization encourages the search to complete a partial program that looks promising, but unlike Probe, offers no guidance on which are the likely productions to complete it with.…”