Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows

Deng, Ruizhi; Chang, Bo; Brubaker, Marcus A.; Mori, Greg; Lehrmann, Andreas M.

doi:10.48550/arxiv.2002.10516

Cited by 5 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SDEs and CDEs are closely related, and several authors have introduced Neural SDEs. [36,37,38] treat them as generative models for time series and seek to model the data distribution. [39,40] investigate using stochasticity as a regularizer, and demonstrate better performance by doing so.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Neural Controlled Differential Equations for Irregular Time Series

Kidger,

Morrill,

Foster

et al. 2020

Preprint

View full text Add to dashboard Cite

Neural ordinary differential equations are an attractive option for modelling temporal dynamics. However, a fundamental issue is that the solution to an ordinary differential equation is determined by its initial condition, and there is no mechanism for adjusting the trajectory based on subsequent observations. Here, we demonstrate how this may be resolved through the well-understood mathematics of controlled differential equations. The resulting neural controlled differential equation model is directly applicable to the general setting of partiallyobserved irregularly-sampled multivariate time series, and (unlike previous work on this problem) it may utilise memory-efficient adjoint-based backpropagation even across observations. We demonstrate that our model achieves state-of-the-art performance against similar (ODE or RNN based) models in empirical studies on a range of datasets. Finally we provide theoretical results demonstrating universal approximation, and that our model subsumes alternative ODE models.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Modelling uncertainty As presented here, Neural CDEs do not give any measure of uncertainty about their predictions. Such extensions are likely to be possible, given the close links between CDEs and SDEs, and existing work on Neural SDEs [36,37,38,39,40,42,43,44].…”

Section: Future Workmentioning

confidence: 99%

Neural Controlled Differential Equations for Irregular Time Series

Kidger,

Morrill,

Foster

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…We use an EMF with continuity or smoothness structure. The baselines are MAF and MAF-L models like in the previous sections, and an MAF in which the base distribution follows the continuity structure (B-MAF), as a discrete-time version of the model proposed in (Deng et al, 2020). We then use GEMF-T with either continuity or smoothness.…”

Section: Timeseries Modelingmentioning

confidence: 99%

Embedded-model flows: Combining the inductive biases of model-free deep learning and explicit probabilistic modeling

Silvestri¹,

Fertig²,

Moore³

et al. 2021

Preprint

View full text Add to dashboard Cite

Normalizing flows have shown great success as general-purpose density estimators. However, many real world applications require the use of domain-specific knowledge, which normalizing flows cannot readily incorporate. We propose embedded-model flows (EMF), which alternate general-purpose transformations with structured layers that embed domain-specific inductive biases. These layers are automatically constructed by converting user-specified differentiable probabilistic models into equivalent bijective transformations. We also introduce gated structured layers, which allow bypassing the parts of the models that fail to capture the statistics of the data. We demonstrate that EMFs can be used to induce desirable properties such as multimodality, hierarchical coupling and continuity. Furthermore, we show that EMFs enable a high performance form of variational inference where the structure of the prior model is embedded in the variational architecture. In our experiments, we show that this approach outperforms state-of-the-art methods in common structured inference problems.

show abstract

“…Some works in computer graphics have used the interpolation effect of CNFs to model transformations of point clouds (Yang et al, 2019;Rempe et al, 2020;. CNFs have also been used in sequential latent variable models (Deng et al, 2020;Rempe et al, 2020). However, such works do not align the "time" axis of the CNF with the temporal axis of observations, and do not train on observations at more than one value of "time" in the CNF.…”

Section: Continuous Normalizing Flowsmentioning

confidence: 99%

Neural Spatio-Temporal Point Processes

Chen¹,

Amos²,

Nickel³

2020

Preprint

View full text Add to dashboard Cite

We propose a new class of parameterizations for spatio-temporal point processes which leverage Neural ODEs as a computational method and enable flexible, highfidelity models of discrete events that are localized in continuous time and space. Central to our approach is a combination of recurrent continuous-time neural networks with two novel neural architectures, i.e., Jump and Attentive Continuoustime Normalizing Flows. This approach allows us to learn complex distributions for both the spatial and temporal domain and to condition non-trivially on the observed event history. We validate our models on data sets from a wide variety of contexts such as seismology, epidemiology, urban mobility, and neuroscience.

show abstract

Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows

Cited by 5 publications

References 16 publications

Neural Controlled Differential Equations for Irregular Time Series

Neural Controlled Differential Equations for Irregular Time Series

Embedded-model flows: Combining the inductive biases of model-free deep learning and explicit probabilistic modeling

Neural Spatio-Temporal Point Processes

Contact Info

Product

Resources

About