Towards Reproducibility in Scientific Workflows: An Infrastructure-Based Approach

Abstract

It is commonly agreed that in silico scientific experiments should be executable and repeatable processes. Most of the current approaches for computational experiment conservation and reproducibility have focused so far on two of the main components of the experiment, namely, data and method. In this paper, we propose a new approach that addresses the third cornerstone of experimental reproducibility: the equipment. This work focuses on the equipment of a computational experiment, that is, the set of software and hardware components that are involved in the execution of a scientific workflow. In order to demonstrate the feasibility of our proposal, we describe a use case scenario on the Text Analytics domain and the application of our approach to it. From the original workflow, we document its execution environment, by means of a set of semantic models and a catalogue of resources, and generate an equivalent infrastructure for reexecuting it.

The Experiment Factory: Reproducible Experiment Containers

“…n.d.), Baker (n.d.), Open Science Collaboration (2015)) has been well met by many efforts (Belmann et al 2015, Moreews et al (2015, Boettiger (2014), Santana-Perez and Pérez-Hernández (2015), Wandell et al (2015)) across scientific disciplines to capture dependencies required for a scientific analysis. Behavioral research is especially challenging, historically due to the need to bring a study participant into the lab, and currently due to needing to develop and validate a well-tested set of paradigms.…”

A characterization of workflow management systems for extreme-scale applications

“…Many workflows provide mechanisms for tracing provenance and methodologies that foster reproducible science [31].…”

Reproducibility Analysis of Scientific Workflows

“…Santana-Perez et al [19] proposed an alternative approach to reproduce scientific workflows which focused on the equipment of a computational experiment. They have developed an infrastructure-aware approach for computational execution environment conservation and reproducibility based on documenting the components of the infrastructure.…”