Inductive program synthesis over noisy data

Handa, Shivam; Rinard, Martin

doi:10.1145/3368089.3409732

Cited by 15 publications

(26 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rose is parameterized over a large class of objective functions, loss functions, and complexity measures. [Handa and Rinard 2020] highlights that this flexibility is required to synthesize correct programs for datasets which contain a large amount of noise.…”

Section: Noisy Program Synthesis Using Abstractions 1:3mentioning

confidence: 99%

“…We first review the noisy program synthesis framework (introduced by Rinard 2021, 2020]), the concepts associated with this framework, and the conditions that qualify a program to be the correct solution to a synthesis problem. We also discuss the tree automata based noisy program synthesis technique proposed by [Handa and Rinard 2020].…”

Section: Preliminariesmentioning

confidence: 99%

“…This line of work aims to tackle real world datasets which contain noise and corruptions. These techniques can synthesize the correct programs, even in presence of substantial noise [Handa and Rinard 2020].…”

Section: Introductionmentioning

confidence: 99%

“…Working with noisy datasets further complicates the search: synthesizing a program which simply maximizes the number of input-output examples it satisfies (is correct on) may not optimize the objective function over the entire dataset. Moreover, recent research has demonstrated that noisy program synthesis can often synthesize the correct program even when all input/output examples are corrupted [Handa and Rinard 2020].…”

Section: Introductionmentioning

confidence: 99%

“…Current solutions to noisy program synthesis, either fall into enumeration based search techniques [Peleg and Polikarpova 2020] or version space (finite tree automaton) based techniques [Handa and Rinard 2020]. Both of these techniques reduce the search space by partitioning the space of programs based on their execution behaviour on the given inputs.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Inductive Program Synthesis over Noisy Datasets using Abstraction Refinement Based Optimization

Handa¹,

Rinard²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We present a new synthesis algorithm to solve program synthesis over noisy datasets, i.e., data that may contain incorrect/corrupted input-output examples. Our algorithm uses an abstraction refinement based optimization process to synthesize programs which optimize the tradeoff between the loss over the noisy dataset and the complexity of the synthesized program. The algorithm uses abstractions to divide the search space of programs into subspaces by computing an abstract value that represents outputs for all programs in a subspace. The abstract value allows our algorithm to compute, for each subspace, a sound approximate lower bound of the loss over all programs in the subspace. It iteratively refines these abstractions to further subdivide the space into smaller subspaces, prune subspaces that do not contain an optimal program, and eventually synthesize an optimal program.We implemented this algorithm in a tool called Rose. We compare Rose to a current state-of-the-art noisy program synthesis system [Handa and Rinard 2020] using the SyGuS 2018 benchmark suite [Alur et al. 2013]. Our evaluation demonstrates that Rose significantly outperforms this previous system: on two noisy benchmark program synthesis problem sets drawn from the SyGus 2018 benchmark suite, Rose delivers speedups of up to 1587 and 81.7, with median speedups of 20.5 and 81.7. Rose also terminates on 20 (out of 54) and 4 (out of 11) more benchmark problems than the previous system. Both Rose and the previous system synthesize programs that are optimal over the provided noisy data sets. For the majority of the problems in the benchmark sets (272 out of 286), the synthesized programs also produce correct outputs for all inputs in the original (unseen) noise-free data set. These results highlight the benefits that Rose can deliver for effective noisy program synthesis.

show abstract

Section: Noisy Program Synthesis Using Abstractions 1:3mentioning

confidence: 99%

Section: Preliminariesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Inductive Program Synthesis over Noisy Datasets using Abstraction Refinement Based Optimization

Handa¹,

Rinard²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Towards Reliable SQL Synthesis: Fuzzing-Based Evaluation and Disambiguation

Brancas,

Terra-Neves,

Ventura

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In recent years, more people have seen their work depend on data manipulation tasks. However, many of these users do not have the background in programming required to write complex programs, particularly SQL queries. One way of helping these users is automatically synthesizing the SQL query given a small set of examples. Several program synthesizers for SQL have been recently proposed, but they do not leverage multicore architectures.This paper proposes Cubes, a parallel program synthesizer for the domain of SQL queries using input-output examples. Since input-output examples are an under-specification of the desired SQL query, sometimes, the synthesized query does not match the user’s intent. Cubes incorporates a new disambiguation procedure based on fuzzing techniques that interacts with the user and increases the confidence that the returned query matches the user intent. We perform an extensive evaluation on around 4000 SQL queries from different domains. Experimental results show that our parallel approach can scale up to 16 processes with super-linear speedups for many hard instances, and that our disambiguation approach is critical to achieving an accuracy of around 60%, significantly larger than other SQL synthesizers.

show abstract