Program synthesis by type-guided abstraction refinement

Abstract

We consider the problem of type-directed component-based synthesis where, given a set of (typed) components and a query type, the goal is to synthesize a term that inhabits the query. Classical approaches based on proof search in intuitionistic logics do not scale up to the standard libraries of modern languages, which span hundreds or thousands of components. Recent graph reachability based methods proposed for Java do scale, but only apply to monomorphic data and components: polymorphic data and components infinitely explode the size of the graph that must be searched, rendering synthesis intractable. We introduce type-guided abstraction refinement (TYGAR), a new approach for scalable type-directed synthesis over polymorphic datatypes and components. Our key insight is that we can overcome the explosion by building a graph over abstract types which represent a potentially unbounded set of concrete types. We show how to use graph reachability to search for candidate terms over abstract types, and introduce a new algorithm that uses proofs of untypeability of ill-typed candidates to iteratively refine the abstraction until a well-typed result is found. We have implemented TYGAR in H+, a tool that takes as input a set of Haskell libraries and a query type, and returns a Haskell term that uses functions from the provided libraries to implement the query type. Our support for polymorphism allows H+ to work with higher-order functions and type classes, and enables more precise queries due to parametricity. We have evaluated H+ on 44 queries using a set of popular Haskell libraries with a total of 291 components. H+ returns an interesting solution within the first five results for 32 out of 44 queries. Our results show that TYGAR allows H+ to rapidly return well-typed terms, with the median time to first solution of just 1.4 seconds. Moreover, we observe that gains from iterative refinement over exhaustive enumeration are more pronounced on harder queries. CCS Concepts: • Theory of computation → Automated reasoning; • Software and its engineering → Automatic programming.

Program sketching with live bidirectional evaluation

“…To scale to much larger programs with complex control flow, static reasoning (interleaved with concrete evaluation) could be used to prune unsatisfiable or heuristically łdifficultž sets of example constraints. Orthogonal techniques for scaling to large contexts with additional components [Feng et al 2017b;Guo et al 2020;Gvero et al 2013] might also be incorporated into our approach in future work.…”

SyRust: automatic testing of Rust libraries with semantic-aware program synthesis

“…Prior works dealt with polymorphism by either exhaustively instantiating type variables beforehand (purely eager) [14] or keeping all APIs polymorphic and refining them incrementally (purely lazy) [15]. Our observation is that neither purely lazy nor purely eager approaches are sufficient for Rust.…”

“…Leveraging advances in constraint-solving, componentbased program synthesis techniques [19] have been applied to many languages. Most closely related to ours, are SyPet [14] (Java) and H+ [15] (Haskell), which use constraintsolving-based technique to generate loop-free straight-line code. Both SyPet and H+ employ a graph-based encoding, which is unwieldy for encoding the lifetime and ownership of variables; and therefore SyRust does not use.…”

“…The Rust compiler will reject a large portion of programs that are generated using traditional random search techniques like JCrasher [12], Randoop [27] and RESTler [7]. Instead, we take inspiration from the recent success of constraint solving based program synthesis techniques like H+ [15] and SyPet [14]. These techniques take in a set of API specifications and synthesize valid programs by solving a constraint formula that encodes the set of all bounded-length programs that can be built using the given APIs.…”

Programming with a read-eval-synth loop

“…Polikarpova and Sergey [2019] use separation logic proof steps to limit the space to programs that follow the proof. Feng et al [2017a]; Guo et al [2019] use a Petri-net of type signatures to limit the space to reachable paths in the net.…”