Quantitative computational models play an increasingly important role in modern biology. Such models typically involve many free parameters, and assigning their values is often a substantial obstacle to model development. Directly measuring in vivo biochemical parameters is difficult, and collectively fitting them to other experimental data often yields large parameter uncertainties. Nevertheless, in earlier work we showed in a growth-factor-signaling model that collective fitting could yield well-constrained predictions, even when it left individual parameters very poorly constrained. We also showed that the model had a “sloppy” spectrum of parameter sensitivities, with eigenvalues roughly evenly distributed over many decades. Here we use a collection of models from the literature to test whether such sloppy spectra are common in systems biology. Strikingly, we find that every model we examine has a sloppy spectrum of sensitivities. We also test several consequences of this sloppiness for building predictive models. In particular, sloppiness suggests that collective fits to even large amounts of ideal time-series data will often leave many parameters poorly constrained. Tests over our model collection are consistent with this suggestion. This difficulty with collective fits may seem to argue for direct parameter measurements, but sloppiness also implies that such measurements must be formidably precise and complete to usefully constrain many model predictions. We confirm this implication in our growth-factor-signaling model. Our results suggest that sloppy sensitivity spectra are universal in systems biology models. The prevalence of sloppiness highlights the power of collective fits and suggests that modelers should focus on predictions rather than on parameters.

Profiling candidate therapeutics with limited cancer models during preclinical development hinders predictions of clinical efficacy and identifying factors that underlie heterogeneous patient responses for patient-selection strategies. We established ∼1,000 patient-derived tumor xenograft models (PDXs) with a diverse set of driver mutations. With these PDXs, we performed in vivo compound screens using a 1 × 1 × 1 experimental design (PDX clinical trial or PCT) to assess the population responses to 62 treatments across six indications. We demonstrate both the reproducibility and the clinical translatability of this approach by identifying associations between a genotype and drug response, and established mechanisms of resistance. In addition, our results suggest that PCTs may represent a more accurate approach than cell line models for assessing the clinical potential of some therapeutic modalities. We therefore propose that this experimental paradigm could potentially improve preclinical evaluation of treatment modalities and enhance our ability to predict clinical trial responses.

Quantitative computational models play an increasingly important role in modern biology. Such models typically involve many free parameters, and assigning their values is often a substantial obstacle to model development. Directly measuring in vivo biochemical parameters is difficult, and collectively fitting them to other experimental data often yields large parameter uncertainties. Nevertheless, in earlier work we showed in a growth-factor-signaling model that collective fitting could yield well-constrained predictions, even when it left individual parameters very poorly constrained. We also showed that the model had a ''sloppy'' spectrum of parameter sensitivities, with eigenvalues roughly evenly distributed over many decades. Here we use a collection of models from the literature to test whether such sloppy spectra are common in systems biology. Strikingly, we find that every model we examine has a sloppy spectrum of sensitivities. We also test several consequences of this sloppiness for building predictive models. In particular, sloppiness suggests that collective fits to even large amounts of ideal time-series data will often leave many parameters poorly constrained. Tests over our model collection are consistent with this suggestion. This difficulty with collective fits may seem to argue for direct parameter measurements, but sloppiness also implies that such measurements must be formidably precise and complete to usefully constrain many model predictions. We confirm this implication in our growth-factor-signaling model. Our results suggest that sloppy sensitivity spectra are universal in systems biology models. The prevalence of sloppiness highlights the power of collective fits and suggests that modelers should focus on predictions rather than on parameters.

In a variety of contexts, physicists study complex, nonlinear models with many unknown or tunable parameters to explain experimental data. We explain why such systems so often are sloppy; the system behavior depends only on a few 'stiff' combinations of the parameters and is unchanged as other 'sloppy' parameter combinations vary by orders of magnitude. We contrast examples of sloppy models (from systems biology, variational quantum Monte Carlo, and common data fitting) with systems which are not sloppy (multidimensional linear regression, random matrix ensembles). We observe that the eigenvalue spectra for the sensitivity of sloppy models have a striking, characteristic form, with a density of logarithms of eigenvalues which is roughly constant over a large range. We suggest that the common features of sloppy models indicate that they may belong to a common universality class. In particular, we motivate focusing on a Vandermonde ensemble of multiparameter nonlinear models and show in one limit that they exhibit the universal features of sloppy models.PACS numbers: 02.30. Zz, 87.15.Aa, 87.16.Xa, Systems with many parameters are often sloppy. For practical purposes their behavior depends only on a few stiffly constrained combinations of the parameters; other directions in parameter space can change by orders of magnitude without significantly changing the behavior. Given a suitable cost C(p) measuring the change in system behavior as the parameters p vary from their original values p (0) (e.g., a sum of squared residuals), the stiff and sloppy directions can be quantified as eigenvalues and eigenvectors of the Hessian of the cost:. Figure 1 shows the eigenvalues of the cost Hessian for many different systems; those in (a), (b), (c), (d) and (h) are all sloppy. The sensitivity of model behavior to changes along an eigenvector is given by the square root of the eigenvalue-therefore the range in eigenvalues of roughly one million for the sloppy models means that one must change parameters along the sloppiest eigendirection a thousand times more than along the stiffest eigendirection in order to change the behavior by the same amount. Although anharmonic effects rapidly become important along sloppy eigendirections, a principal component analysis of a Monte-Carlo sampling of lowcost states has a similar spectrum of eigenvalues [1]; the sloppy eigendirections become curved sloppy manifolds in parameter space. Similar sloppy behavior has been demonstrated in fourteen systems biology models taken from the literature [2,3], and in three multiparameter interatomic potentials fit to electronic structure data [4]. In these disparate models we see a common, peculiar behavior: the nth stiffest eigendirection is more important than the (n+1)th by a roughly constant factor, giving a total range of eigenvalues of typically over a million for any model with more than eight parameters. We call systems exhibiting these characteristic features sloppy models.This sloppiness has a number of important implications. In estimating predic...