Computational Reproducibility: The Elephant in the Room

Hatton, Les; Genuchten, Michiel van

doi:10.1109/ms.2018.2883805

Cited by 8 publications

(24 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The latter means to start from a published figure (or table) and trace it back to the original data, including the complete provenance and history of the processing steps involved. 50,51 Putting aside such aspects as long-term archival, unified access to data, and availability of the required software including the underlying operating system and compatible hardware, 17 we focus here on design rules of a modular framework taking care of the provenance of eventually published data, i.e. the history of processing steps from the raw data up to the final representation and publication.…”

Section: Reproduciblementioning

confidence: 99%

“…A basic requirement for software, particularly in science, should be correct and functioning code that is flexible and comprehensible. 17 This requires rigorous tests of the software during its development. The idea of tests is, however, not to prove the program to work correctly, as this is mostly impossible based on logi-cal grounds.…”

Section: Reliablementioning

confidence: 99%

“…Tests aim at finding errors in existing software. 17 While the importance of tests is not disputed and often highlighted, 56,57,59,74,[83][84][85] testing software is a difficult matter. 86 This led some prominent professional programmers with decades of experience to inverting the process of writing functional code and corresponding tests.…”

Section: Reliablementioning

confidence: 99%

“…[13][14][15] This poses a severe problem to science, 16 rendering the results obtained this way eventually unscientific. 17 Nowadays, each of the steps two to five involve some help of computers and often programming. For details see the text.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

ASpecD: A Modular Framework for the Analysis of Spectroscopic Data Focussing on Reproducibility and Good Scientific Practice

Popp

Biskup

2021

Preprint

View full text Add to dashboard Cite

Reproducibility is at the heart of science. However, most published results usually lack the information necessary to be independently reproduced. Even more, most authors will not be able to reproduce the results from a few years ago due to lacking a gap-less record of every processing and analysis step including all parameters involved. There is only one way to overcome this problem: developing robust tools for data analysis that, while maintaining a maximum of flexibility in their application, allow the user to perform advanced processing steps in a scientifically sound way. At the same time, the only viable approach for reproducible and traceable analysis is to relieve the user of the responsibility for logging all processing steps and their parameters. This can only be achieved by using a system that takes care of these crucial though often neglected tasks. Here, we present a solution to this problem: a framework for the analysis of spectroscopic data (ASpecD) written in the Python programming language that can be used without any actual programming needed. This framework is made available open-source and free of charge and focusses on usability, small footprint and modularity while ensuring reproducibility and good scientific practice. Furthermore, we present a set of best practices and design rules for scientific software development and data analysis. Together, this empowers scientists to focus on their research minimising the need to implement complex software tools while ensuring full reproducibility. We anticipate this to have a major impact on reproducibility and good scientific practice, as we raise the awareness of their importance, summarise proven best practices and present a working user-friendly software solution.

show abstract

Section: Reproduciblementioning

confidence: 99%

Section: Reliablementioning

confidence: 99%

Section: Reliablementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

ASpecD: A Modular Framework for the Analysis of Spectroscopic Data Focussing on Reproducibility and Good Scientific Practice

Popp

Biskup

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…These are referred to here as the reproducibility deliverables and are available at and also at . Each reproducibility deliverable allows all results, tables and diagrams to be reproduced individually for that study, as well as performing verification checks on machine environment, availability of essential open-source packages, quality of arithmetic and regression testing of the outputs [45].…”

Section: Data Accessibilitymentioning

confidence: 99%

Strong evidence of an information-theoretical conservation principle linking all discrete systems

Hatton

Warr

2019

R. Soc. open sci.

Self Cite

View full text Add to dashboard Cite

Diverse discrete systems share common global properties that lack a unifying theoretical explanation. However, constraining the simplest measure of total information (Hartley–Shannon) in a statistical mechanics framework reveals a principle, the conservation of Hartley–Shannon information (CoHSI) that directly predicts both known and unsuspected common properties of discrete systems, as borne out in the diverse systems of computer software, proteins and music. Discrete systems fall into two categories distinguished by their structure: heterogeneous systems in which there is a distinguishable order of assembly of the system’s components from an alphabet of unique tokens (e.g. proteins assembled from an alphabet of amino acids), and homogeneous systems in which unique tokens are simply binned, counted and rank ordered. Heterogeneous systems are characterized by an implicit distribution of component lengths, with sharp unimodal peak (containing the majority of components) and a power-law tail, whereas homogeneous systems reduce naturally to Zipf’s Law but with a drooping tail in the distribution. We also confirm predictions that very long components are inevitable for heterogeneous systems; that discrete systems can exhibit simultaneously both heterogeneous and homogeneous behaviour; and that in systems with more than one consistent token alphabet (e.g. digital music), the alphabets themselves show a power-law relationship.

show abstract

ASpecD: A Modular Framework for the Analysis of Spectroscopic Data Focussing on Reproducibility and Good Scientific Practice**

Popp

Biskup

2022

Chemistry Methods

View full text Add to dashboard Cite

Reproducibility is at the heart of science. However, most published results usually lack the information necessary to be independently reproduced. Even more, most authors will not be able to reproduce the results from a few years ago due to lacking a gap‐less record of every processing and analysis step including all parameters involved. There is only one way to overcome this problem: developing robust tools for data analysis that, while maintaining a maximum of flexibility in their application, allow the user to perform advanced processing steps in a scientifically sound way. At the same time, the only viable approach for reproducible and traceable analysis is to relieve the user of the responsibility for logging all processing steps and their parameters. This can only be achieved by using a system that takes care of these crucial though often neglected tasks. Here, we present a solution to this problem: a framework for the analysis of spectroscopic data (ASpecD) written in the Python programming language that can be used without any actual programming needed. This framework is made available open‐source and free of charge and focusses on usability, small footprint and modularity while ensuring reproducibility and good scientific practice. Furthermore, we present a set of best practices and design rules for scientific software development and data analysis. Together, this empowers scientists to focus on their research minimising the need to implement complex software tools while ensuring full reproducibility. We anticipate this to have a major impact on reproducibility and good scientific practice, as we raise the awareness of their importance, summarise proven best practices and present a working user‐friendly software solution.

show abstract

Computational Reproducibility: The Elephant in the Room

Cited by 8 publications

References 13 publications

ASpecD: A Modular Framework for the Analysis of Spectroscopic Data Focussing on Reproducibility and Good Scientific Practice

ASpecD: A Modular Framework for the Analysis of Spectroscopic Data Focussing on Reproducibility and Good Scientific Practice

Strong evidence of an information-theoretical conservation principle linking all discrete systems

ASpecD: A Modular Framework for the Analysis of Spectroscopic Data Focussing on Reproducibility and Good Scientific Practice**

Contact Info

Product

Resources

About