Efficient parameterization of large-scale dynamic models based on relative measurements

Abstract

Motivation: Mechanistic models of biochemical reaction networks facilitate the quantitative understanding of biological processes and the integration of heterogeneous datasets.However, some biological processes require the consideration of comprehensive reaction networks and therefore large-scale models. Parameter estimation for such models poses great challenges, in particular when the data are on a relative scale.Results: Here, we propose a novel hierarchical approach combining (i) the efficient analytic evaluation of optimal scaling, offset, and error model parameters with (ii) the scalable evaluation of objective function gradients using adjoint sensitivity analysis. We evaluate the properties of the methods by parameterizing a pan-cancer ordinary differential equation model (>1000 state variables, >4000 parameters) using relative protein, phospho-protein and viability measurements. The hierarchical formulation improves optimizer performance considerably. Furthermore, we show that this approach allows estimating error model parameters with negligible computational overhead when no experimental estimates are available, providing an unbiased way to weight heterogeneous data. Overall, our hierarchical formulation is applicable to a wide range of models, and allows for the efficient parameterization of largescale models based on heterogeneous relative measurements.Contact: jan.hasenauer@helmholtz-muenchen.de Supplementary information: Supplementary information are available at bioRxiv online.Supplementary code and data are available online at

Challenges in the calibration of large-scale ordinary differential equation models

“…Leaving aside the even more complicated task of network inference, parameter optimization and uncertainty analysis are currently the key challenges for large-scale models, for which no satisfactory approaches exist. Due to the high computation times, toolboxes suitable for computing clusters are necessary and have recently been developed (Penas et al, 2017;Schmiester et al, 2019). Moreover, new approaches have to be explored, such as transferring the concept of mini batching from the field of deep learning to optimization (Goodfellow et al, 2016) or MCMC sampling (Seita et al, 2017) of dynamic models, and must be adapted to ODE models.…”

“…As methods which are tailored a problem class tend to outperform black box solutions and since parameter estimation of ODE models is a bounded field, accounting for the specific structure can lead to substantial improvements (Froehlich et al, 2018;Schmiester et al, 2019). Studies have to be performed, which aim at a better understanding of the properties of this problem class, such as how non-identifiable parameters translate into uncertainties of model predictions.…”

“…Among others, estimating parameters in logarithmic scale (Hass et al, 2019) generally improves the convexity of the objective function and optimizer performance. Moreover, due to the specific form of the objective function, observation specific parameters such as scaling or measurement noise parameters can be computed analytically for a given model simulation, which also increases optimizer performance (Loos et al, 2018;Schmiester et al, 2019). In comparative studies of optimization strategies, meta-heuristics combined with local searches or naive multi-start local optimization have shown to be well suited for this class of problems (Raue et al, 2013;Villaverde et al, 2018).…”

Mini-batch optimization enables training of ODE models on large-scale datasets

“…To benchmark our local optimization methods, we used the interior-point optimizer Ipopt (64), which combines a limited-memory BFGS scheme with a line-search approach (44). In previous studies (54), since such interior-point optimizers have shown to be among the most competitive methods for local optimization of large-scale ODE systems (54; 62).…”

“…Parameter estimation was performed using the parPE C++ library (54), which provides the means for parallelized objective function evaluation and optimization of ODE models generated by the AMICI ODE-solver toolbox (16) using optimizers such as Ipopt (64). In our studies, we used Ipopt version 3.12.9, running with linear solver ma27 and L-BFGS approximation of the Hessian matrix and extended parPE with the mini-batch algorithms described above.…”