Data-driven precondition inference with learned features

Padhi, Saswat; Sharma, Rahul; Millstein, Todd

doi:10.1145/2908080.2908099

Cited by 63 publications

(34 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although prior SyGuS tools run multiple instances of learners with different random seeds [7,20], to our knowledge, this is the first proposal to explore multiple grammars as a means to improve the performance of SyGuS. Our experiments indicate that PLearn significantly improves the performance of five state-of-the-art SyGuS tools-CVC4 [7,33], EUSolver [5], LoopInvGen [29], SketchAC [20,37], and Stoch…”

Section: Introductionmentioning

confidence: 85%

“…-LoopInvGen [29] combines enumeration and Boolean function learning. We ran these five tools on 180 invariant-synthesis benchmarks, which we describe in Sect.…”

Section: Grammar Sensitivity Of Sygus Toolsmentioning

confidence: 99%

“…To better understand this phenomenon, we instrumented LoopInvGen [29] to record the candidate expressions that it synthesizes and the number of CEGIS iterations (called rounds henceforth). We compare each pair of successful runs of each of our 180 benchmarks on distinct grammars.…”

Section: Evidence For Overfittingmentioning

confidence: 99%

“…Given these two inputs, a SyGuS tool searches through the programs that are permitted by the grammar to generate one that meets the specification. Today, SyGuS is at the core of several state-of-the-art program synthesizers [5,14,23,24,29], many of which compete annually in the SyGuS competition [1,4].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Overfitting in Synthesis: Theory and Practice

Padhi

Millstein

Nori

et al. 2019

Computer Aided Verification

Self Cite

View full text Add to dashboard Cite

In syntax-guided synthesis (SyGuS), a synthesizer's goal is to automatically generate a program belonging to a grammar of possible implementations that meets a logical specification. We investigate a common limitation across state-of-the-art SyGuS tools that perform counterexample-guided inductive synthesis (CEGIS). We empirically observe that as the expressiveness of the provided grammar increases, the performance of these tools degrades significantly.We claim that this degradation is not only due to a larger search space, but also due to overfitting. We formally define this phenomenon and prove no-free-lunch theorems for SyGuS, which reveal a fundamental tradeoff between synthesizer performance and grammar expressiveness.A standard approach to mitigate overfitting in machine learning is to run multiple learners with varying expressiveness in parallel. We demonstrate that this insight can immediately benefit existing SyGuS tools. We also propose a novel single-threaded technique called hybrid enumeration that interleaves different grammars and outperforms the winner of the SyGuS competition (Inv track), solving more problems and achieving a 5× mean speedup. *

show abstract

Section: Introductionmentioning

confidence: 85%

“…-LoopInvGen [29] combines enumeration and Boolean function learning. We ran these five tools on 180 invariant-synthesis benchmarks, which we describe in Sect.…”

Section: Grammar Sensitivity Of Sygus Toolsmentioning

confidence: 99%

Section: Evidence For Overfittingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Overfitting in Synthesis: Theory and Practice

Padhi

Millstein

Nori

et al. 2019

Computer Aided Verification

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is our future work to integrate the idea of ICE learning with our graph-based learning. The work in [38] presents an approach for precondition inference. The main contribution is feature learning for functional programs.…”

Section: Related Workmentioning

confidence: 99%

Compositional Verification of Heap-Manipulating Programs Through Property-Guided Learning

Pham

Sun

2019

Programming Languages and Systems

View full text Add to dashboard Cite

Analyzing and verifying heap-manipulating programs automatically is challenging. A key for fighting the complexity is to develop compositional methods. For instance, many existing verifiers for heap-manipulating programs require user-provided specification for each function in the program in order to decompose the verification problem. The requirement, however, often hinders the users from applying such tools. To overcome the issue, we propose to automatically learn heap-related program invariants in a property-guided way for each function call. The invariants are learned based on the memory graphs observed during test execution and improved through memory graph mutation. We implemented a prototype of our approach and integrated it with two existing program verifiers. The experimental results show that our approach enhances existing verifiers effectively in automatically verifying complex heap-manipulating programs with multiple function calls.

show abstract

Code2Inv: A Deep Learning Framework for Program Verification

Naik

Dai

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We propose a general end-to-end deep learning framework Code2Inv, which takes a verification task and a proof checker as input, and automatically learns a valid proof for the verification task by interacting with the given checker. Code2Inv is parameterized with an embedding module and a grammar: the former encodes the verification task into numeric vectors while the latter describes the format of solutions Code2Inv should produce. We demonstrate the flexibility of Code2Inv by means of two small-scale yet expressive instances: a loop invariant synthesizer for C programs, and a Constrained Horn Clause (CHC) solver.

show abstract

Data-driven precondition inference with learned features

Cited by 63 publications

References 34 publications

Overfitting in Synthesis: Theory and Practice

Overfitting in Synthesis: Theory and Practice

Compositional Verification of Heap-Manipulating Programs Through Property-Guided Learning

Code2Inv: A Deep Learning Framework for Program Verification

Contact Info

Product

Resources

About