Unbiased estimation of log normalizing constants with applications to Bayesian cross-validation

Rischard, Maxime; Jacob, Pierre E.; Pillai, Natesh

doi:10.48550/arxiv.1810.01382

Cited by 10 publications

(12 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…return (f t , J t ) 10: end function 11: (z, ∆ log p ω t ) ← odesolve(aug,(x, 0),0,T ) 12: return log p 0 (z) − ∆ log p ω t A number of techniques exist for debiasing the logarithm of ( 22) -see Rhee & Glynn (2015) and Rischard et al (2018), for example. Alternatively, we lie in the setting of semi-implicit variational inference seen in Yin & Zhou (2018) and Titsias & Ruiz (2019), and those techniques directly extend to our case as well.…”

Section: Resultsmentioning

confidence: 99%

Stochastic Normalizing Flows

Hodgkinson,

van der Heide,

Roosta

et al. 2020

Preprint

View full text Add to dashboard Cite

We introduce stochastic normalizing flows, an extension of continuous normalizing flows for maximum likelihood estimation and variational inference (VI) using stochastic differential equations (SDEs). Using the theory of rough paths, the underlying Brownian motion is treated as a latent variable and approximated, enabling efficient training of neural SDEs as random neural ordinary differential equations. These SDEs can be used for constructing efficient Markov chains to sample from the underlying distribution of a given dataset. Furthermore, by considering families of targeted SDEs with prescribed stationary distribution, we can apply VI to the optimization of hyperparameters in stochastic MCMC.

show abstract

Section: Resultsmentioning

confidence: 99%

Stochastic Normalizing Flows

Hodgkinson,

van der Heide,

Roosta

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…This is of interest as from the numerical results, they suggest alternative rates for (F3), different to that of the other variants. One could also provide an unbiased estimation of the NC [36], which has connections to MLMC [9,37]. In order to do so, one would require a modified multilevel analysis of the EnKBF, where one has uniform upper bounds, with respect to levels l = 1, .…”

Section: Discussionmentioning

confidence: 99%

“…From (1.1) -(1.2), V t and W t are independent d y and d x −dimensional Brownian motions respectively, with h : R dx → R dy and f : R dx → R dx denoting potentially nonlinear functions, and σ : R dx → R dx×dx acting as a diffusion coefficient. Aside from computing the filtering distribution, filtering can also be exploited to compute normalizing constants associated with the filtering distribution [7,8,21,27,36], i.e. the marginal likelihood, which is an important and useful computation in Bayesian statistics.…”

Section: Introductionmentioning

confidence: 99%

Multilevel Estimation of Normalization Constants Using the Ensemble Kalman-Bucy Filter

Ruzayqat¹,

Chada²,

Jasra³

2021

Preprint

View full text Add to dashboard Cite

In this article we consider the application of multilevel Monte Carlo, for the estimation of normalizing constants. In particular we will make use of the filtering algorithm, the ensemble Kalman-Bucy filter (EnKBF), which is an N -particle representation of the Kalman-Bucy filter (KBF). The EnKBF is of interest as it coincides with the optimal filter in the continuous-linear setting, i.e. the KBF. This motivates our particular setup in the linear setting. The resulting methodology we will use is the multilevel ensemble Kalman-Bucy filter (MLEnKBF). We provide an analysis based on deriving Lqbounds for the normalizing constants using both the single-level, and the multilevel algorithms. Our results will be highlighted through numerical results, where we firstly demonstrate the error-to-cost rates of the MLEnKBF comparing it to the EnKBF on a linear Gaussian model. Our analysis will be specific to one variant of the MLEnKBF, whereas the numerics will be tested on different variants. We also exploit this methodology for parameter estimation, where we test this on the models arising in atmospheric sciences, such as the stochastic Lorenz 63 and 96 model.

show abstract

“…To reduce implementation effort, it is sometimes possible to introduce a path of distributions (π t ) so that only slight modifications to the existing MCMC algorithm are required to target each bridging distribution. We describe a concrete example taken from Rischard et al [2018].…”

Section: Path Of Least Coding Effortmentioning

confidence: 99%

An invitation to sequential Monte Carlo samplers

Dai¹,

Heng²,

Jacob³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Sequential Monte Carlo samplers provide consistent approximations of sequences of probability distributions and of their normalizing constants, via particles obtained with a combination of importance weights and Markov transitions. This article presents this class of methods and a number of recent advances, with the goal of helping statisticians assess the applicability and usefulness of these methods for their purposes. Our presentation emphasizes the role of bridging distributions for computational and statistical purposes. Numerical experiments are provided on simple settings such as multivariate Normals, logistic regression and a basic susceptible-infected-recovered model, illustrating the impact of the dimension, the ability to perform inference sequentially and the estimation of normalizing constants.

show abstract

Unbiased estimation of log normalizing constants with applications to Bayesian cross-validation

Cited by 10 publications

References 32 publications

Stochastic Normalizing Flows

Stochastic Normalizing Flows

Multilevel Estimation of Normalization Constants Using the Ensemble Kalman-Bucy Filter

An invitation to sequential Monte Carlo samplers

Contact Info

Product

Resources

About