Synthesizing bijective lenses

Abstract

Bidirectional transformations between different data representations occur frequently in modern software systems. They appear as serializers and deserializers, as parsers and pretty printers, as database views and view updaters, and as a multitude of different kinds of ad hoc data converters. Manually building bidirectional transformationsÐby writing two separate functions that are intended to be inversesÐis tedious and error prone. A better approach is to use a domain-specific language in which both directions can be written as a single expression. However, these domain-specific languages can be difficult to program in, requiring programmers to manage fiddly details while working in a complex type system.We present an alternative approach. Instead of coding transformations manually, we synthesize them from declarative format descriptions and examples. Specifically, we present Optician, a tool for type-directed synthesis of bijective string transformers. The inputs to Optician are a pair of ordinary regular expressions representing two data formats and a few concrete examples for disambiguation. The output is a well-typed program in Boomerang (a bidirectional language based on the theory of lenses). The main technical challenge involves navigating the vast program search space efficiently. In particular, and unlike most prior work on type-directed synthesis, our system operates in the context of a language with a rich equivalence relation on types (the theory of regular expressions). Consequently, program synthesis requires search in two dimensions: First, our synthesis algorithm must find a pair of łsyntactically compatible types,ž and second, using the structure of those types, it must find a type-and example-compliant term. Our key insight is that it is possible to reduce the size of this search space without losing any computational power by defining a new language of lenses designed specifically for synthesis. The new language is free from arbitrary function composition and operates only over types and terms in a new disjunctive normal form. We prove (1) our new language is just as powerful as a more natural, compositional, and declarative language and (2) our synthesis algorithm is sound and complete with respect to the new language. We also demonstrate empirically that our new language changes the synthesis problem from one that admits intractable solutions to one that admits highly efficient solutions, able to synthesize intricate lenses between complex file formats in seconds. We evaluate Optician on a benchmark suite of 39 examples that includes both microbenchmarks and realistic examples derived from other data management systems including Flash Fill, a tool for synthesizing string transformations in spreadsheets, and Augeas, a tool for bidirectional processing of Linux system configuration files.

Synthesizing differentially private programs

“…Our work follows the tradition of program synthesis for various data-manipulation tasks, e.g., [Gulwani et al 2012;Le and Gulwani 2014,? ;Miltner et al 2018;Polozov and Gulwani 2015;Smith and Albarghouthi 2016;Wang et al 2017a,b;Yaghmazadeh et al 2017;Zhang and Sun 2013]. Perhaps the most closely related work to ours is Bigλ [Smith and Albarghouthi 2016], which targets data-analysis programs composed of higher-order combinators like map and reduce.…”

Narcissus: correct-by-construction derivation of decoders and encoders from binary formats

“…While Boomerang originally synthesized functions that assumed that every source value had a canonical target representation, it has since been extended with quotient lenses that relax this restriction . The recently developed Optician tool [Miltner et al 2017] synthesizes Boomerang programs that implement bijective string transformations from regular expressions describing source and target formats and sets of user-provided disambiguating examples. The format-decoding problem differs from the lens setting in that lenses consider how to recover a new source value from an updated target value given full knowledge of the old source value, while decoding must work given only a single target value.…”

Syntax-Guided Termination Analysis

“…SyGuS [2] is applied to various tasks related to program synthesis, e.g., [13,17,25,33,35,41]. However, the formal grammar in those applications is typically given or constructed from user-provided examples.…”