Implementing code review in the scientific workflow: Insights from ecology and evolutionary biology

Ivimey-Cook, Edward R.; Pick, Joel L.; Bairos-Novak, Kevin R.; Culina, Antica; Gould, Elliot; Grainger, Matthew; Marshall, Benjamin M.; Moreau, David; Paquet, Matthieu; Royauté, Raphaël; Sánchez-Tójar, Alfredo; Silva, Inês; Windecker, Saras M.

doi:10.1111/jeb.14230

Cited by 11 publications

(3 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While widespread availability of code would undoubtedly assist audit studies investigating computational reproducibility post-publication, the success rate of such studies would be further improved (perhaps substantially so) if code was reviewed before publication, perhaps as part of peer review as discussed by Fernández-Juricic [ 67 ]. Ivimey-Cook et al [ 68 ] provide a comprehensive primer of code review at all stages of a research project, outlining a workflow for conducting effective reviews. Implementing code review into the research process (whether as part of formal peer review or not) would require a change in current research practices and the allocation of resources; the costs of this would need to be compared against the advantages of enhancing the reproducibility of reported results.…”

Section: Discussionmentioning

confidence: 99%

Computationally reproducing results from meta-analyses in ecology and evolutionary biology using shared code and data

Kambouris,

Wilkinson,

Smith

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

Many journals in ecology and evolutionary biology encourage or require authors to make their data and code available alongside articles. In this study we investigated how often this data and code could be used together, when both were available, to computationally reproduce results published in articles. We surveyed the data and code sharing practices of 177 meta-analyses published in ecology and evolutionary biology journals published between 2015–17: 60% of articles shared data only, 1% shared code only, and 15% shared both data and code. In each of the articles which had shared both (n = 26), we selected a target result and attempted to reproduce it. Using the shared data and code files, we successfully reproduced the targeted results in 27–73% of the 26 articles, depending on the stringency of the criteria applied for a successful reproduction. The results from this sample of meta-analyses in the 2015–17 literature can provide a benchmark for future meta-research studies gauging the computational reproducibility of published research in ecology and evolutionary biology.

show abstract

Section: Discussionmentioning

confidence: 99%

Computationally reproducing results from meta-analyses in ecology and evolutionary biology using shared code and data

Kambouris,

Wilkinson,

Smith

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…In line with our philosophy that simply uploading code to a repository is only the first step towards replicable methods, the MIROSLAV analysis code is predominantly written in the form of notebooks [45][46][47][48] -documents with human-readable, formatted text serving as documentation, with interspersed code cells where configuration parameters can be modified and code can be executed to produce the desired results within the document itself. With this approach, users of all programming skill levels can quickly become intimately familiar with how their data is handled throughout the entire process.…”

Section: Mirosinementioning

confidence: 99%

My friend MIROSLAV: A Hackable Open-Source Hardware and Software Platform for High-Throughput Rodent Activity Monitoring in the Home Cage

Virag,

Homolak,

Kodvanj

et al. 2024

Preprint

View full text Add to dashboard Cite

While conventional behavioural tests offer valuable insight into rodent behaviour in specific paradigms, rodents spend most of their time in a safe, undisturbed environment – their home cage. However, home cage monitoring (HCM) is often impractical at scale over prolonged periods of time without significant loss of data.To achieve this, we developed MIROSLAV, the Multicage InfraRed Open Source Locomotor Activity eValuator, a home cage activity monitoring device designed to be open, modular, and robust. Its transparent and modular design allows the device to be tailored to varying experimental requirements and environmental conditions, while its wireless operation and multiple redundancies minimise loss of data. Data quality is maintained by a modular software workflow for data preparation (Prepare-a-SLAV) and cleaning (TidySLAV), followed by exploratory (MIROSine – MIRO The Explorer) and statistical (MIROSine – StatistiSLAV) analysis of circadian periodicity.Here, using MIROSLAV, we demonstrate circadian dysrhythmia and a disrupted response to regular stimuli (e.g. behavioural testing, bedding change) in a rat model of sporadic Alzheimer’s disease, showcasing MIROSLAV’s utility in a typical animal study of disease.In accordance with the 3Rs, every rodent study and laboratory performing them can benefit from HCM, provided the existence of customisable, modular tools which allow robust deployment to tens or hundreds of cages in various conditions. MIROSLAV provides this opportunity to researchers in a cost-effective manner.

show abstract

“…There are a series of good practices in biodiversity data management that should be followed in similar studies but evaluating other taxa or regions. First, it is important to document the use of scripts and publish the code (either in R or in other languages) in open repositories such as GitHub, to make the whole process comparable and reproducible (Ivimey‐Cook et al., 2023 ). Second, researchers should review the most recent packages and web services to manage occurrence records and use the most updated source of taxonomy available (Grenié et al., 2022 ).…”

Section: Main Conclusionmentioning

confidence: 99%

Exploring the impact of data curation criteria on the observed geographical distribution of mosses

Ronquillo,

Stropp,

Medina

et al. 2023

Ecology and Evolution

View full text Add to dashboard Cite

Biodiversity data records contain inaccuracies and biases. To overcome this limitation and establish robust geographic patterns, ecologists often curate records keeping those that are most suitable for their analyses. Yet, this choice is not straightforward and the outcome of the analysis may vary due to a trade‐off between data quality and volume. This problem is particularly recurrent for less‐studied groups with patchy sampling effort. The latitudinal pattern of mosses richness remains inconsistent across studies and these may emerge purely from sampling artefacts. Our main objective here is to assess the effect of different curation criteria on this spatial pattern in the Temperate Northern Hemisphere (above 20° latitude). We contrasted the geographical distribution of moss species records and the latitude‐species richness relation obtained under different data curation scenarios. These scenarios comprehend five sources of taxonomical standardisations and eight data cleaning filters. The analyses are based on the selection of well‐surveyed cells at 100 km cell resolution. The application of some ‘data curation scenarios’ severely affects the number of records selected for analysis and substantially changes the proportion of richness per cell. The sensitivity to data curation becomes detectable at regional and at the cell scales showing a large shift in the latitudinal richness peak in Europe, from 60° N to 45° N latitude, when only preserved specimens are selected and duplicates based on date of collection and coordinates are excluded. Our results stress the importance of justifying the criteria used for filtering biodiversity data retrieved from biodiversity databases to avoid detecting misleading patterns. Curating records under particular criteria compromises the information in some areas displaying different spatial information of mosses. This problem can be ameliorated if data filtering is combined with identifying well‐surveyed cells, render relatively constant results under different combinations of filtering even for less well‐known groups such as mosses.

show abstract

Implementing code review in the scientific workflow: Insights from ecology and evolutionary biology

Cited by 11 publications

References 51 publications

Computationally reproducing results from meta-analyses in ecology and evolutionary biology using shared code and data

Computationally reproducing results from meta-analyses in ecology and evolutionary biology using shared code and data

My friend MIROSLAV: A Hackable Open-Source Hardware and Software Platform for High-Throughput Rodent Activity Monitoring in the Home Cage

Exploring the impact of data curation criteria on the observed geographical distribution of mosses

Contact Info

Product

Resources

About