Toward practical transparent verifiable and long-term reproducible research using Guix

Abstract

Reproducibility crisis urge scientists to promote transparency which allows peers to draw same conclusions after performing identical steps from hypothesis to results. Growing resources are developed to open the access to methods, data and source codes. Still, the computational environment, an interface between data and source code running analyses, is not addressed. Environments are usually described with software and library names associated with version labels or provided as an opaque container image. This is not enough to describe the complexity of the dependencies on which they rely to operate on. We describe this issue and illustrate how open tools like Guix can be used by any scientist to share their environment and allow peers to reproduce it. Some steps of research might not be fully reproducible, but at least, transparency for computation is technically addressable. These tools should be considered by scientists willing to promote transparency and open science.

The five pillars of computational reproducibility: bioinformatics and beyond

“…Although regular software updates are overall a good thing, it poses a problem for future reproducibility. A researcher trying to reproduce a 10-year-old study in R v3.0 could have a headache, as they would need to roll back their R version, and possibly their operating system as well, as R and other languages require certain system dependencies for low-level routines [ 84 ].…”

“…Guix is described as a ‘functional package manager’ and has the unique benefit of giving bit-for-bit build reproducibility and verifiability [ 84 , 97 ]. This is a notable advantage, as Docker images are not verifiable nor guaranteed to build reproducibly in the future due to link decay.…”

The five pillars of computational reproducibility: Bioinformatics and beyond

“…Although regular software updates are overall a good thing, it poses a problem for future reproducibility. A researcher trying to reproduce a tenyear-old study in R v3.0 could have a headache, as they would need to roll back their R version, and possibly their operating system as well, as R and other languages require certain system dependencies for low-level routines [85].…”

“…Guix is described as a "functional package manager" and has the unique benefit of giving bit-for-bit build reproducibility and verifiability [85,98]. This is a notable advantage, as Docker images are not verifiable nor guaranteed to build reproducibly in the future due to link decay.…”

Parallel Empirical Evaluations: Resilience despite Concurrency

“…For algorithm configuration, pitfalls in empirical work have been previously identified (Bocchese et al 2018) and best practices to avoid them suggested (Eggensperger, Lindauer, and Hutter 2019). Recent initiatives on reproducible research focus on transparent research artifacts with Guix, a system that enables the building of computation environments (Vallet, Michonneau, and Tournier 2022). Software heritage projects aim at preserving source code and binaries from research software (Cosmo and Zacchiroli 2017;Audemard, Paulevé, and Simon 2020).…”