Fabular: regression formulas as probabilistic programming

Abstract

Regression formulas are a domain-specific language adopted by several R packages for describing an important and useful class of statistical models: hierarchical linear regressions. Formulas are succinct, expressive, and clearly popular, so are they a useful addition to probabilistic programming languages? And what do they mean? We propose a core calculus of hierarchical linear regression, in which regression coefficients are themselves defined by nested regressions (unlike in R). We explain how our calculus captures the essence of the formula DSL found in R. We describe the design and implementation of Fabular, a version of the Tabular schema-driven probabilistic programming language, enriched with formulas based on our regression calculus. To the best of our knowledge, this is the first formal description of the core ideas of R's formula notation, the first development of a calculus of regression formulas, and the first demonstration of the benefits of composing regression formulas and latent variables in a probabilistic programming language.

Tabular: Probabilistic Inference from the Spreadsheet

“…The dissertation also described a new, arguably more rigorous and elegant, semantics of Core Tabular models. Fabular, presented by Borgström et al (2016), extends Tabular with hierarchical linear regression formulas, extending the formula notation used by R packages such as lmer. Such formulas allow for a concise representation of a wide class of models and can be used in Tabular like any other model expressions.…”

From high-level inference algorithms to efficient code

“…-Elimination is widely applied to discrete and continuous variables [de Salvo Braz et al 2007;Dechter 1998;Poole and Zhang 2003;Sanner and Abbasnejad 2012;Poole 1994, 1996] and is known in various contexts as Rao-Blackwellization [Blackwell 1947;Casella and Robert 1996;Gelfand and Smith 1990;Kolmogorov 1950;Murray et al 2018;Rao 1945], collapse [Koller and Friedman 2009;Venugopal and Gogate 2013], marginalization [Meng and van Dyk 1999;Obermeyer et al 2018], and integrating out [Griffiths and Steyvers 2004;Resnik and Hardisty 2010]. -Conjugacy is a preferred starting point and basic building block of Bayesian data modeling [Gelman et al 2014, page 36] and underlies such popular applications as Naive Bayes classification [Bayes 1763] and Bayesian linear regression [Borgström et al 2016]. • Loop optimization includes reordering sums to achieve superlinear speedups, and fusing and specializing loops to obtain one more order of magnitude in performance.…”

“…collapse [Koller and Friedman 2009;Venugopal and Gogate 2013], marginalization [Meng and van Dyk 1999;Obermeyer et al 2018], and integrating out [Griffiths and Steyvers 2004;Resnik and Hardisty 2010]. -Conjugacy is a preferred starting point and basic building block of Bayesian data modeling [Gelman et al 2014, page 36] and underlies such popular applications as Naive Bayes classification [Bayes 1763] and Bayesian linear regression [Borgström et al 2016]. • Loop optimization includes reordering sums to achieve superlinear speedups, and fusing and specializing loops to obtain one more order of magnitude in performance.…”