On the relationship between sloppiness and identifiability

Abstract

Dynamic models of biochemical networks are often formulated as sets of non-linear ordinary differential equations, whose states are the concentrations or abundances of the network components. They typically have a large number of kinetic parameters, which must be determined by calibrating the model with experimental data. In recent years it has been suggested that dynamic systems biology models are universally sloppy, meaning that the values of some parameters can be perturbed by several orders of magnitude without causing significant changes in the model output. This observation has prompted calls for focusing on model predictions rather than on parameters. In this work we examine the concept of sloppiness, investigating its links with the long-established notions of structural and practical identifiability. By analysing a set of case studies we show that sloppiness is not equivalent to lack of identifiability, and that sloppy models can be identifiable. Thus, using sloppiness to draw conclusions about the possibility of estimating parameter values can be misleading. Instead, structural and practical identifiability analyses are better tools for assessing the confidence in parameter estimates. Furthermore, we show that, when designing new experiments to decrease parametric uncertainty, designs that optimize practical identifiability criteria are more informative than those that minimize sloppiness.

Performance of objective functions and optimisation procedures for parameter estimation in system biology models

“…Therefore, we analysed practical (a posteriori) identifiability. 36 Briefly, by analysing the parameter estimates from multiple runs, we assess practical non-identifiability. If the parameter estimates are largely variable in a certain direction, then this direction in the parameter space (e.g.…”

“…However, using a linear approximation, the dimensionality of the non-identifiable manifold can be estimated using principal component analysis (PCA). For details, we refer to Gutenkunst et al 36, 37 . Briefly, PCA gives optimal directions in the parameter space, so-called principal components (PCs) that can best explain variability of the parameter estimates.…”

“…37 If all parameters are variable, the system is not “sloppy”, yet it can be unidentifiable if the variance is large 36, 45 ).…”

Dynamic compensation, parameter identifiability, and equivariances

“…Another classical example is furnished by proteases acting in a close to zeroth-order regime. This notion of “almost independence” is also studied in the identification literature, under names such as “practical identifiability” (or more precisely, lack thereof) and is closely related to (though not exactly the same as) the concept of “sloppy” models in systems biology [28, 29, 30, 14, 15, 31]. When applied in the “dynamical compensation” context, these notions should be useful in understanding weaker notions of almost-robustness of homeostatic control systems in physiology.…”

The best models of metabolism

“…In addition to not being able to obtain good solutions, parameter estimation can lead to entire domains of combinations of parameter values that yield essentially equivalent solutions. This issue of sloppiness has been discussed frequently in recent times (e.g., Refs ). Thus, in some cases, there are no good solutions, and in other cases, there are arguably too many.…”