A call for virtual experiments: Accelerating the scientific process

Abstract

Experimentation is fundamental to the scientific method, whether for exploration, description or explanation. We argue that promoting the reuse of virtual experiments (the in silico analogues of wet-lab or field experiments) would vastly improve the usefulness and relevance of computational models, encouraging critical scrutiny of models and serving as a common language between modellers and experimentalists. We review the benefits of reusable virtual experiments: in specifying, assaying, and comparing the behavioural repertoires of models; as prerequisites for reproducible research; to guide model reuse and composition; and for quality assurance in the translational application of models. A key step towards achieving this is that models and experimental protocols should be represented separately, but annotated so as to facilitate the linking of models to experiments and data. Lastly, we outline how the rigorous, streamlined confrontation between experimental datasets and candidate models would enable a "continuous integration" of biological knowledge, transforming our approach to systems biology. Dear Sirs,We enclose a manuscript titled "A call for virtual experiments: accelerating the scientific process" for inclusion within the Special Issue on "Multi-bio and multi-scale systems biology" in Progress in Biophysics and Molecular Biology. In it we propose the concept of "virtual experiments" as the in silico analogues of wet-lab or field experiments, and argue that this concept deserves as much attention as the computational models to which such experiments are applied. Promoting the reuse of virtual experiments would vastly improve the usefulness and relevance of computational models, encouraging critical scrutiny of models and serving as a common language between modellers and experimentalists.Experiments are fundamental to the scientific method, whether for exploration, description or explanation. Experiments should therefore be as central in computational biology as in other sciences, for studying the implications of theories or hypotheses encoded as computational models. In current practice, however, experiments applied to models are usually embedded in opaque computer code, which is often not available for public scrutiny. We discuss how the protocol for a computational experiment should be viewed as an entity in its own right, and cleanly separated from the description of the model itself, and from surrounding code. Annotations can then facilitate the linking of models to experiment descriptions and any corresponding data available.This approach can provide many benefits for scientific progress: in specifying, assaying, and comparing the behavioural repertoires of models; as a prerequisite for reproducible research; to guide model reuse and composition; and for quality assurance in the application of computational biology models. The paper lays out these benefits to motivate wider adoption and tool support for virtual experiments. We also present a "vision of the future" in which virtual experim...

Reproducibility in Computational Neuroscience Models and Simulations

“…Curated BioModels [146] entries have the annotations embedded in the SBML files themselves. These annotations typically map model parameters and equations to biological concepts, but they may also map the model itself to the dataset that inspired it [147]. Related information may or may not be included in a model's associated publication, but putting all of the annotation in a machine readable form allows tools like the NIF LinkOut Broker [148] to automatically find complementary information and resources.…”

“…MIASE standards can also be satisfied by sharing code, assuming adequate code transparency. For example, making sure to separate model description from simulation procedure maintains the distinction between the above categories, thereby improving clarity and reproducibility [147]. …”

Extracting reproducible simulation studies from model repositories using the CombineArchive Toolkit

“…For example, Research Objects [BDR + 10] support the publication of data, code and other resources alongside the PDF paper. A call for Virtual Experiments [CVW14] highlights the actual benefits of generic simulation setups for the reusability of full behavioural repertoires of computational models.…”

Epithelial Modelling Platform: A Tool for Model Discovery and Assembly with the Physiome Model Repository

“…A recent initiative in (Cooper et al, 2015a), the Cardiac Electrophysiology Web Lab, has introduced an online simulation environment to compare simulation experiments. Such virtual experiments are analogous to in silico environment of in vivo and in vitro experiments, or wet lab experiments (Cooper et al, 2015b).…”