Maximal multi-layer specification synthesis

Abstract

There has been a significant interest in applying programming-byexample to automate repetitive and tedious tasks. However, due to the incomplete nature of input-output examples, a synthesizer may generate programs that pass the examples but do not match the user intent. In this paper, we propose Mars, a novel synthesis framework that takes as input a multi-layer specification composed by inputoutput examples, textual description, and partial code snippets that capture the user intent. To accurately capture the user intent from the noisy and ambiguous description, we propose a hybrid model that combines the power of an LSTM-based sequence-tosequence model with the apriori algorithm for mining association rules through unsupervised learning. We reduce the problem of solving a multi-layer specification synthesis to a Max-SMT problem, where hard constraints encode well-typed concrete programs and soft constraints encode the user intent learned by the hybrid model. We instantiate our hybrid model to the data wrangling domain and compare its performance against Morpheus, a state-of-the-art synthesizer for data wrangling tasks. Our experiments demonstrate that our approach outperforms Morpheus in terms of running time and solved benchmarks. For challenging benchmarks, our approach can suggest candidates with rankings that are an order of magnitude better than Morpheus which leads to running times that are 15x faster than Morpheus. CCS CONCEPTS • Software and its engineering → Programming by example; Automatic programming.

FOREST: An Interactive Multi-tree Synthesizer for Regular Expressions

“…A model that satisfies the formula represents a valid regex. Due to space constraints we omit the k-tree encoding but further details can be found in the literature [2,17].…”

“…Program synthesis has been successfully used in many domains such as string processing [8,19,7,26], query synthesis [11,25,17], data wrangling [2,5], and functional synthesis [3,6]. In this section, we discuss prior work on the synthesis of regular expressions [10,1] that is most closely related to our approach.…”

Program Synthesis Using Deduction-Guided Reinforcement Learning

“…Over the past decade, there has been significant interest in automatically synthesizing programs from high-level expressions of user intent [2,6,21,23,25,39,40,46]. Some of these techniques are geared towards computer end-users and therefore utilize informal specifications such as inputoutput examples [23,40,50], natural language [24,42,55,56], or a combination of both [10,12]. On the other hand, program synthesis techniques geared towards programmers often utilize additional information, such as a program sketch [17,36,46,49] or types [33,39] in addition to test cases [20,30] or logical specifications [6,49].…”

“…This paper is related to a long line of work on using machine learning for program synthesis. Among these techniques, some of them train a machine learning model (typically a deep neural network) to directly predict a full program from the given specification [12,16,34,35]. Many of these approaches are based on sequence-to-sequence models [47], sequence to tree models [56], or graph neural networks [41] commonly used in machine translation.…”

HISyn: human learning-inspired natural language programming

“…Lin [28] leverages recurrent neural networks (RNNs) for NL to code translation. Chen [5] applies LSTM-based sequence-to-sequence model with other specifications for code synthesis. Applying these approaches to program analysis would require many training examples to cover the vast space of possible code complexities and situations.…”