Continualization of Probabilistic Programs With Correction

Laurel, Jacob; Misailovíc, Saša

doi:10.1007/978-3-030-44914-8_14

Cited by 6 publications

(2 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, such cases can be statically detected and the program can be transformed into one that uses the equivalent condition as 𝑜𝑢𝑡 > 0.5, so that H2 holds. More in general, continuity corrections such as those performed in Laurel and Misailovic [2020] can be adopted.…”

Section: Satisfaction Of the Hypothesesmentioning

confidence: 99%

Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures

Randone,

Bortolussi,

Incerto

et al. 2024

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Computing the posterior distribution of a probabilistic program is a hard task for which no one-fit-for-all solution exists. We propose Gaussian Semantics, which approximates the exact probabilistic semantics of a bounded program by means of Gaussian mixtures. It is parametrized by a map that associates each program location with the moment order to be matched in the approximation. We provide two main contributions. The first is a universal approximation theorem stating that, under mild conditions, Gaussian Semantics can approximate the exact semantics arbitrarily closely. The second is an approximation that matches up to second-order moments analytically in face of the generally difficult problem of matching moments of Gaussian mixtures with arbitrary moment order. We test our second-order Gaussian approximation (SOGA) on a number of case studies from the literature. We show that it can provide accurate estimates in models not supported by other approximation methods or when exact symbolic techniques fail because of complex expressions or non-simplified integrals. On two notable classes of problems, namely collaborative filtering and programs involving mixtures of continuous and discrete distributions, we show that SOGA significantly outperforms alternative techniques in terms of accuracy and computational time.

show abstract

Section: Satisfaction Of the Hypothesesmentioning

confidence: 99%

Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures

Randone,

Bortolussi,

Incerto

et al. 2024

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

show abstract

“…Another closely related PL work by aims to establish Lipschitz robustness of programs, but they use the classical Jacobian and do not support AD. Follow-up works showed how to analyze Lipschitz robustness of non-differentiable programs by analyzing their smooth approximation [Chaudhuri and Solar-Lezama 2011], stemming from the fact that several PL works focus on differentiable approximations of non-differentiable programs and ML models [Chaudhuri and Solar-Lezama 2010;Laurel and Misailovic 2020]. In contrast, DeepJ can analyze Lipschitz properties of programs with points of non-differentiability directly without using approximations, by employing the more general Clarke Jacobian.…”

Section: Related Workmentioning

confidence: 99%

A dual number abstraction for static analysis of Clarke Jacobians

Laurel

Yang

Singh

et al. 2022

Proc. ACM Program. Lang.

Self Cite

View full text Add to dashboard Cite

We present a novel abstraction for bounding the Clarke Jacobian of a Lipschitz continuous, but not necessarily differentiable function over a local input region. To do so, we leverage a novel abstract domain built upon dual numbers, adapted to soundly over-approximate all first derivatives needed to compute the Clarke Jacobian. We formally prove that our novel forward-mode dual interval evaluation produces a sound, interval domain-based over-approximation of the true Clarke Jacobian for a given input region. Due to the generality of our formalism, we can compute and analyze interval Clarke Jacobians for a broader class of functions than previous works supported – specifically, arbitrary compositions of neural networks with Lipschitz, but non-differentiable perturbations. We implement our technique in a tool called DeepJ and evaluate it on multiple deep neural networks and non-differentiable input perturbations to showcase both the generality and scalability of our analysis. Concretely, we can obtain interval Clarke Jacobians to analyze Lipschitz robustness and local optimization landscapes of both fully-connected and convolutional neural networks for rotational, contrast variation, and haze perturbations, as well as their compositions.

show abstract

Continualization of Probabilistic Programs With Correction

Laurel

Misailovíc

2020

Programming Languages and Systems

Self Cite

View full text Add to dashboard Cite

Probabilistic Programming offers a concise way to represent stochastic models and perform automated statistical inference. However, many real-world models have discrete or hybrid discrete-continuous distributions, for which existing tools may suffer non-trivial limitations. Inference and parameter estimation can be exceedingly slow for these models because many inference algorithms compute results faster (or exclusively) when the distributions being inferred are continuous. To address this discrepancy, this paper presents Leios. Leios is the first approach for systematically approximating arbitrary probabilistic programs that have discrete, or hybrid discrete-continuous random variables. The approximate programs have all their variables fully continualized. We show that once we have the fully continuous approximate program, we can perform inference and parameter estimation faster by exploiting the existing support that many languages offer for continuous distributions. Furthermore, we show that the estimates obtained when performing inference and parameter estimation on the continuous approximation are still comparably close to both the true parameter values and the estimates obtained when performing inference on the original model.

show abstract

Continualization of Probabilistic Programs With Correction

Cited by 6 publications

References 53 publications

Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures

Inference of Probabilistic Programs with Moment-Matching Gaussian Mixtures

A dual number abstraction for static analysis of Clarke Jacobians

Continualization of Probabilistic Programs With Correction

Contact Info

Product

Resources

About