A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions

Rackauckas, Christopher; Ma, Yingbo; Dixit, Vaibhav; Guo, Xiaojing; Innes, Mike; Revels, Jarrett; Nyberg, Joakim; Ivaturi, Vijay

doi:10.1109/hpec49654.2021.9622796

Cited by 40 publications

(31 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The computational complexity to get the derivatives is in O ( np ) where n is the number of time steps and p is the number of parameters 24 . While the scaling of the forward AD is the same as the FD, the forward AD is faster than FD for small ODE systems (<100 parameters) as illustrated in the referenced article 24 because forward AD calculates derivatives simultaneously with function evaluations. In the reverse AD, the derivative of θ n +1 with respect to the LHS is back propagated to the variables in the RHS.…”

Section: Methodsmentioning

confidence: 99%

Evaluation of the degree of rate control via automatic differentiation

2022

View full text Add to dashboard Cite

The degree of rate control (DRC) quantitatively identifies the kinetically relevant (sometimes known as rate‐limiting) steps of a complex reaction network. This concept relies on derivatives which are commonly implemented numerically, for example, with finite differences (FDs). Numerical derivatives are tedious to implement, and can be problematic, and unstable or unreliable. In this study, we demonstrate the use of automatic differentiation (AD) in the evaluation of the DRC. AD libraries are increasingly available through modern machine learning frameworks. Compared with the FDs, AD provides solutions with higher accuracy with lower computational cost. We demonstrate applications in steady‐state and transient kinetics. Furthermore, we illustrate a hybrid local‐global sensitivity analysis method, the distributed evaluation of local sensitivity analysis, to assess the importance of kinetic parameters over an uncertain space. This method also benefits from AD to obtain high‐quality results efficiently.

show abstract

Section: Methodsmentioning

confidence: 99%

Evaluation of the degree of rate control via automatic differentiation

2022

View full text Add to dashboard Cite

show abstract

“…Another technique one can use to calculate the derivatives of model solutions is to differentiate the numerical algorithm that calculates the solution. This can be done with computational tools collectively known as automatic differentiation [19]. Forward mode automatic differentiation is performed by carrying forward Jacobian-vector products at each successive calculation.…”

Section: Automatic Differentiationmentioning

confidence: 99%

Differential methods for assessing sensitivity in biological models

et al. 2022

Self Cite

View full text Add to dashboard Cite

Differential sensitivity analysis is indispensable in fitting parameters, understanding uncertainty, and forecasting the results of both thought and lab experiments. Although there are many methods currently available for performing differential sensitivity analysis of biological models, it can be difficult to determine which method is best suited for a particular model. In this paper, we explain a variety of differential sensitivity methods and assess their value in some typical biological models. First, we explain the mathematical basis for three numerical methods: adjoint sensitivity analysis, complex perturbation sensitivity analysis, and forward mode sensitivity analysis. We then carry out four instructive case studies. (a) The CARRGO model for tumor-immune interaction highlights the additional information that differential sensitivity analysis provides beyond traditional naive sensitivity methods, (b) the deterministic SIR model demonstrates the value of using second-order sensitivity in refining model predictions, (c) the stochastic SIR model shows how differential sensitivity can be attacked in stochastic modeling, and (d) a discrete birth-death-migration model illustrates how the complex perturbation method of differential sensitivity can be generalized to a broader range of biological models. Finally, we compare the speed, accuracy, and ease of use of these methods. We find that forward mode automatic differentiation has the quickest computational time, while the complex perturbation method is the simplest to implement and the most generalizable.

show abstract

“…In the following sections we demonstrate the approach on two problems. This work is coded in Julia [18], using the following packages: DifferentialEquations.jl [19], [20], DiffEqFlux [2], [17], Flux.jl [21], ForwardDiff.jl [22], NLopt.jl [23] and Hyperopt.jl [24].…”

Section: Multiple Shooting With Neural Desmentioning

confidence: 99%

Multiple Shooting for Training Neural Differential Equations on Time Series

Turan

Jäschke

2022

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

Neural differential equations have recently emerged as a flexible data-driven/hybrid approach to model time-series data. This work experimentally demonstrates that if the data contains oscillations, then standard fitting of a neural differential equation may result in a "flattened out" trajectory that fails to describe the data. We then introduce the multiple shooting method and present successful demonstrations of this method for the fitting of a neural differential equation to two datasets (synthetic and experimental) that the standard approach fails to fit. Constraints introduced by multiple shooting can be satisfied using a penalty or augmented Lagrangian method.

show abstract

A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions

Cited by 40 publications

References 19 publications

Evaluation of the degree of rate control via automatic differentiation

Evaluation of the degree of rate control via automatic differentiation

Differential methods for assessing sensitivity in biological models

Multiple Shooting for Training Neural Differential Equations on Time Series

Contact Info

Product

Resources

About