Bayesian synthesis of probabilistic programs for automatic data modeling

Abstract

We present new techniques for automatically constructing probabilistic programs for data analysis, interpretation, and prediction. These techniques work with probabilistic domain-specific data modeling languages that capture key properties of a broad class of data generating processes, using Bayesian inference to synthesize probabilistic programs in these modeling languages given observed data. We provide a precise formulation of Bayesian synthesis for automatic data modeling that identifies sufficient conditions for the resulting synthesis procedure to be sound. We also derive a general class of synthesis algorithms for domain-specific languages specified by probabilistic context-free grammars and establish the soundness of our approach for these languages. We apply the techniques to automatically synthesize probabilistic programs for time series data and multivariate tabular data. We show how to analyze the structure of the synthesized programs to compute, for key qualitative properties of interest, the probability that the underlying data generating process exhibits each of these properties. Second, we translate probabilistic programs in the domain-specific language into probabilistic programs in Venture, a general-purpose probabilistic programming system. The translated Venture programs are then executed to obtain predictions of new time series data and new multivariate data records. Experimental results show that our techniques can accurately infer qualitative structure in multiple real-world data sets and outperform standard data analysis methods in forecasting and predicting new data.

Elements of a stochastic 3D prediction engine in larval zebrafish prey capture

“…DPMMs can approximate a broad class of multivariate distributions without requiring a priori specification of the number of components in the mixture model. The mixture models generated via a DPMM prior can be converted to probabilistic programs for inference to generate the kinds of conditional simulations used in Figure 6 (Saad et al, 2019). In this representation, each pre-bout to post-bout prey transformation made by a zebrafish can be thought of as arising from a program that first chooses a prototypical transform (corresponding to a component in the mixture), and then generates a random transform from a distribution over transforms associated with the prototype.…”

Inductive Synthesis for Probabilistic Programs Reaches New Horizons

“…This motivated (1) abstractionrefinement over the MDP representation [10], and (2) counterexample-guided inductive synthesis (CEGIS) for MCs [9], mentioned earlier. The alternative problem of sketching for probabilistic programs that fit given data is studied, e.g., in [32,38].…”

Sound probabilistic inference via guide types

“…For the first question, we obtained 23 benchmarks from prior work [40] and collected 6 new benchmarks. The 29 benchmark programs consist of (i) example models from Anglican [62], Turing [20], and Pyro [8], as well as (ii) PCFG models, including a Gaussian-process domain-specific language (DSL) [50] and synthetic models (such as examples shown in this paper). Compared to prior work [40], a larger subset of benchmark models are expressible and type-checked in our PPL.…”