A variational Bayesian approach for inverse problems with skew-t error distributions

Guha, Nilabja; Wu, Xiaoqing; Efendiev, Yalchin; Jin, Bangti; Mallick, Bani K.

doi:10.1016/j.jcp.2015.07.062

Cited by 17 publications

(10 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gábor and Banga (2014) suggested improving the estimation of ODE parameters using the Tikhonov regularization framework. Additional approaches dealing with numerically solving inverse problems and studying uncertainty quantification in the context of dynamical systems can be found in Jin and Zou (2010), Jin (2012), and Guha, Wu, Efendiev, Jin, and Mallick (2015). In these papers and references therein, techniques like Variational Bayes are used and sparseness properties of the underlying model are exploited.…”

Section: Parameter Estimation Of Odesmentioning

confidence: 99%

Differential equations in data analysis

Dattner

2020

WIREs Computational Stats

View full text Add to dashboard Cite

Differential equations have proven to be a powerful mathematical tool in science and engineering, leading to better understanding, prediction, and control of dynamic processes. In this paper, we review the role played by differential equations in data analysis. More specifically, we consider the intersection between differential equations and data analysis in the light of modern statistical learning methodologies. This article is categorized under: Data: Types and Structure > Time Series, Stochastic Processes, and Functional Data Statistical and Graphical Methods of Data Analysis > Nonparametric Methods Statistical Models > Nonlinear Models

show abstract

Section: Parameter Estimation Of Odesmentioning

confidence: 99%

Differential equations in data analysis

Dattner

2020

WIREs Computational Stats

View full text Add to dashboard Cite

show abstract

“…For the gradient with respect to the Cholesky factor R q , we note that ∂y k /∂R q,ij = δ ki z j . By the chain rule, we have Finally, we have ∇ Rq log det R q = (R −1 q ) , from which we recover (21). The gradients with respect to the prior hyperparameters, (22) can be derived from [9], Eqs.…”

Section: Appendix a Gaussian Backpropagation Rulesmentioning

confidence: 99%

“…Furthermore, both methods are applicable to non-factorizing likelihoods, do not require computing moments of the likelihood, and do not require third-or higher order derivatives of the physics model. Finally, variational inference and the Laplace approximation have been employed for model inversion [19,20,21,22,13], but to the best of our knowledge, have not been used in the context of the empirical Bayesian framework to estimate GP prior hyperparameters, with the exception of the work of [13]. That work, based on the Laplace approximation, requires computing third-order derivatives of the physics model, which may be costly to compute, whereas the presented methods do not require third-order derivatives.…”

Section: Introductionmentioning

confidence: 99%

Approximate Bayesian model inversion for PDEs with heterogeneous and state-dependent coefficients

Barajas-Solano

Tartakovsky

2019

Journal of Computational Physics

View full text Add to dashboard Cite

We present two approximate Bayesian inference methods for parameter estimation in partial differential equation (PDE) models with space-dependent and state-dependent parameters. We demonstrate that these methods provide accurate and cost-effective alternatives to Markov Chain Monte Carlo simulation. We assume a parameterized Gaussian prior on the unknown functions, and approximate the posterior density by a parameterized multivariate Gaussian density. The parameters of the prior and posterior are estimated from sparse observations of the PDE model's states and the unknown functions themselves by maximizing the evidence lower bound (ELBO), a lower bound on the log marginal likelihood of the observations. The first method, Laplace-EM, employs the expectation maximization algorithm to maximize the ELBO, with a Laplace approximation of the posterior on the Estep, and minimization of a Kullback-Leibler divergence on the M-step. The second method, DSVI-EB, employs the doubly stochastic variational inference (DSVI) algorithm, in which the ELBO is maximized via gradient-based stochastic optimization, with nosiy gradients computed via simple Monte Carlo sampling and Gaussian backpropagation. We apply these methods to identifying diffusion coefficients in linear and nonlinear diffusion equations, and we find that both methods provide accurate estimates of posterior densities and the hyperparameters of Gaussian priors. While the Laplace-EM method is more accurate, it requires computing Hessians of the physics model. The DSVI-EB method is found to be less accurate but only requires gradients of the physics model.

show abstract

“…Approximate inference methods motivated by a broader class of applications are in their infancy. A small, growing, literature includes McGrory et al (2009), Gehre and Jin (2014) and Guha et al (2015). Arridge et al (2018) and Zhang et al (2019) respectively study usage of Gaussian variational approximations and expectation propagation to fit inverse problems models with Poisson responses.…”

Section: Introductionmentioning

confidence: 99%

A Variational Inference Framework for Inverse Problems

Maestrini¹,

Aykroyd²,

Wand³

2021

Preprint

View full text Add to dashboard Cite

We present a framework for fitting inverse problem models via variational Bayes approximations. This methodology guarantees flexibility to statistical model specification for a broad range of applications, good accuracy performances and reduced model fitting times, when compared with standard Markov chain Monte Carlo methods. The message passing and factor graph fragment approach to variational Bayes we describe facilitates streamlined implementation of approximate inference algorithms and forms the basis to software development. Such approach allows for supple inclusion of numerous response distributions and penalizations into the inverse problem model. Albeit our analysis is circumscribed to one-and two-dimensional response variables, we lay down an infrastructure where streamlining algorithmic steps based on nullifying weak interactions between variables are extendible to inverse problems in higher dimensions. Image processing applications motivated by biomedical and archaeological problems are included as illustrations.

show abstract

A variational Bayesian approach for inverse problems with skew-t error distributions

Cited by 17 publications

References 21 publications

Differential equations in data analysis

Differential equations in data analysis

Approximate Bayesian model inversion for PDEs with heterogeneous and state-dependent coefficients

A Variational Inference Framework for Inverse Problems

Contact Info

Product

Resources

About