Generative ODE modeling with known unknowns

Abstract

In several crucial applications, domain knowledge is encoded by a system of ordinary differential equations (ODE). A motivating example is intensive care unit patients: The dynamics of some vital physiological variables such as heart rate, blood pressure and arterial compliance can be approximately described by a known system of ODEs. Typically, some of the ODE variables are directly observed while some are unobserved, and in addition many other variables are observed but not modeled by the ODE, for example body temperature. Importantly, the unobserved ODE variables are "known-unknowns": We know they exist and their functional dynamics, but cannot measure them directly, nor do we know the function tying them to all observed measurements. Estimating these known-unknowns is often highly valuable to physicians. Under this scenario we wish to: (i) learn the static parameters of the ODE generating each observed time-series (ii) infer the dynamic sequence of all ODE variables including the known-unknowns, and (iii) extrapolate the future of the ODE variables and the observations of the time-series. We address this task with a variational autoencoder incorporating the known ODE function, called GOKU-net for Generative ODE modeling with Known Unknowns. We test our method on videos of pendulums with unknown length, and a model of the cardiovascular system.

Predicting the impact of treatments over time with uncertainty aware neural differential equations

“…Cardiovascular Model. We use a model of the cardiovascular system proposed in Zenker et al (2007) and Linial et al (2021) to study the capacity of our model to learn the impact of fluids intake. Fluids are commonly administered for treating severe hypotension, but individual patients response is difficult to assess beforehand, making it a very pertinent case study.…”

“…The dosage of the treatment A is set as a function of the the amplitude as well. We use a model of the cardiovascular system as proposed in (Zenker et al, 2007;Linial et al, 2021) and use to study the capacity of our model to learn the impact of fluids intake. Fluid intake is commonly used for treating severe hypotension.…”

Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations

“…Closely related to our work are latent variable state-space models focused on features that separate static from dynamic (Yingzhen and Mandt, 2018;Fraccaro et al, 2017), domain-invariant from domain-specific (Miladinović et al, 2019), position from momentum (Yildiz et al, 2019), and parameter (system input) estimation (Linial et al, 2021). In contrast, our work focuses on synthesizing observational data y(t) from dynamical systems given: (i) combinations of previously unseen inputs u (also known as zero-shot learning), and (ii) a simulated continuous-time state-space x(t) from an ODE solver.…”

“…We further divide methods that learn nonlinear dynamics according to assumptions required for estimating ODE dynamics, where f (•) is modeled as a neural network (Chen et al, 2018), or more recently, parameterized by a latent variable model (Rubanova et al, 2019), that leverages amortized variational inference (Kingma and Welling, 2013;. While a large body of machine learning approaches assume a known parametric form of the dynamics f (•) (Linial et al, 2021;Wan et al, 2001;Wenk et al, 2020), alternative flexible approaches assume that the parametric form of f (•) is unknown (Rubanova et al, 2019;Roeder et al, 2019). Moreover, several specifications of variational inference for latent variable state-space models have been proposed (Linial et al, 2021;Rubanova et al, 2019;Karl et al, 2017;Roeder et al, 2019;Miladinović et al, 2019;Yingzhen and Mandt, 2018;Fraccaro et al, 2017).…”

“…While a large body of machine learning approaches assume a known parametric form of the dynamics f (•) (Linial et al, 2021;Wan et al, 2001;Wenk et al, 2020), alternative flexible approaches assume that the parametric form of f (•) is unknown (Rubanova et al, 2019;Roeder et al, 2019). Moreover, several specifications of variational inference for latent variable state-space models have been proposed (Linial et al, 2021;Rubanova et al, 2019;Karl et al, 2017;Roeder et al, 2019;Miladinović et al, 2019;Yingzhen and Mandt, 2018;Fraccaro et al, 2017). Of these, only Roeder et al (2019) considers a structured hierarchical latent variable model accounting for both observations and system inputs.…”