Variational Inference for Stochastic Differential Equations

Opper, Manfred

doi:10.1002/andp.201800233

Cited by 28 publications

(16 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While sampling based methods of inference for SDE models do exist [64,65], these are generally not scalable to large datasets or to models with many parameters. Instead, we use an approximate variational inference approach [66,67]. We assume a parametric form of the posterior that is optimized to be close to the true posterior.…”

Section: Variational Approximation For Scalable Bayesian Inferencementioning

confidence: 99%

“…The functional form of the posterior drift is both more general and more easily trainable than the network SDE in Eq 4, but ultimately is forced to be close to the network dynamics in Eq (4) by the loss function. The loss function for this approach has been previously derived [66,67]. The imputed baseline states x 0 are averaged over.…”

Section: Variational Approximation For Scalable Bayesian Inferencementioning

confidence: 99%

See 1 more Smart Citation

Interpretable machine learning for high-dimensional trajectories of aging health

et al. 2022

View full text Add to dashboard Cite

We have built a computational model for individual aging trajectories of health and survival, which contains physical, functional, and biological variables, and is conditioned on demographic, lifestyle, and medical background information. We combine techniques of modern machine learning with an interpretable interaction network, where health variables are coupled by explicit pair-wise interactions within a stochastic dynamical system. Our dynamic joint interpretable network (DJIN) model is scalable to large longitudinal data sets, is predictive of individual high-dimensional health trajectories and survival from baseline health states, and infers an interpretable network of directed interactions between the health variables. The network identifies plausible physiological connections between health variables as well as clusters of strongly connected health variables. We use English Longitudinal Study of Aging (ELSA) data to train our model and show that it performs better than multiple dedicated linear models for health outcomes and survival. We compare our model with flexible lower-dimensional latent-space models to explore the dimensionality required to accurately model aging health outcomes. Our DJIN model can be used to generate synthetic individuals that age realistically, to impute missing data, and to simulate future aging outcomes given arbitrary initial health states.

show abstract

Section: Variational Approximation For Scalable Bayesian Inferencementioning

confidence: 99%

Section: Variational Approximation For Scalable Bayesian Inferencementioning

confidence: 99%

Interpretable machine learning for high-dimensional trajectories of aging health

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In its most general setting of sampling from a target distribution, this formulation was known to Dai Pra (1991). Tzen and Raginsky (2019b) study the theoretical properties of this approach in the context of generative models (Kingma et al, 2021;Goodfellow et al, 2014), finally Opper (2019) applies this formulation to time series modelling. In contrast our focus is on the estimation of a Bayesian posterior for a broader class of models than Tzen and Raginsky explore.…”

Section: Stochastic Control Formulationmentioning

confidence: 99%

Bayesian Learning via Neural Schrödinger-Föllmer Flows

Vargas¹,

Ovsianas²,

Fernandes³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this work we explore a new framework for approximate Bayesian inference in large datasets based on stochastic control. We advocate stochastic control as a finite time alternative to popular steady-state methods such as stochastic gradient Langevin dynamics (SGLD). Furthermore, we discuss and adapt the existing theoretical guarantees of this framework and establish connections to already existing VI routines in SDE-based models.

show abstract

“…The theory of information has very practical application in physics and elsewhere. Manfred Opper discusses how the solutions of stochastic differential equations can be inferred via variational methods, which have their origin in thermodynamics . Torsten Enßlin introduces information field theory (IFT), the information theory for fields, as a means to infer the configuration of a physical field from incomplete and noisy data .…”

Section: Informationmentioning

confidence: 99%

The Physics of Information

Enßlin

Jasche

Skilling³

2019

Annalen der Physik

View full text Add to dashboard Cite

InformationWhat is information? A short answer could be: Anything that changes our minds and states of thinking! Information is virtual. It can be carried by speech, an image, scratches on a stone, patterns in a photon field, the connections of neurons in our brains or even by the wavefunction of an electron. Information does not depend on the actual means of transmission nor does its identity change when changing the carrier. Information is physical. It needs to be sustained by a physical substrate and rearranging that substrate to contain the wanted information requires to spend energy or work. Without a physical world, information can not be stored, processed, or transmitted. Our world is permeated with different representations of information. Actually Information about the physical laws governing our Universe can be found everywhere and in everything.Could it then be that the real fundamental elements of this world are tiny bits of information? Could we get it from bit? [1] A number of researcher investigate this possibility, and several of their attempts to recover physical laws from the principles governing information are reported about in this special issue on physics of information. The articles by Gregg Jaeger, [2] by Ariel Caticha, [3] and by John Skilling and Kevin Knuth [4] address the reconstruction of quantum mechanics from information theory. Furthermore, Kevin Knuth and James Walsh [5] show how to recover space-time from particles sending each others signals in an initially geometry-free setting.A central element of most works on information is the concept of a probability. Probabilities can be defined via various routes. Most familiar might be their definitions by the frequentists school as relative frequencies of events (in the limit of infinite amounts of such). A more general definition of probabilities is the Bayesians' one, which regards probabilities as quantifiers of statements, expressing how much those should be expected to be true. Although this definition looks subjective on the first glance, it rests on a solid mathematical theorem by Richard Cox [6] that any extension of binary logic to a continuum of single valued truthfulness quantifier that respects a minimum of requirements must be isomorphic to probability theory. And this definition turns out to embrace the frequentists' definition in case that infinite numbers of comparable events are actually available and can be counted. The majority of authors of this special issues seem to favor the Bayesian perspective, which is probably a result of our own, the guest editors', preference and not necessarily a reflection of the prevailing position of contemporary scientists.The different views on probabilities of frequentists and Bayesians are addressed by the article of Allen Caldwell. [7] He builds a bridge between these antagonistic camps by explaining frequentists' constructions in a Bayesian language. In particular, his argumentation why the omnipresent p-value in statistical and experimental literature might often capture a releva...

show abstract

Variational Inference for Stochastic Differential Equations

Cited by 28 publications

References 31 publications

Interpretable machine learning for high-dimensional trajectories of aging health

Interpretable machine learning for high-dimensional trajectories of aging health

Bayesian Learning via Neural Schrödinger-Föllmer Flows

The Physics of Information

Contact Info

Product

Resources

About