Lessons learned: A neuroimaging research center's transition to open and reproducible science

Abstract

Human functional neuroimaging has evolved dramatically in recent years, driven by increased technical complexity and emerging evidence that functional neuroimaging findings are not generally reproducible. In response to these trends, neuroimaging scientists have developed principles, practices, and tools to both manage this complexity as well as to enhance the rigor and reproducibility of neuroimaging science. We group these best practices under four categories: experiment pre-registration, FAIR data principles, reproducible neuroimaging analyses, and open science. While there is growing recognition of the need to implement these best practices there exists little practical guidance of how to accomplish this goal. In this work, we describe lessons learned from efforts to adopt these best practices within the Brain Imaging Research Center at the University of Arkansas for Medical Sciences over 4 years (July 2018–May 2022). We provide a brief summary of the four categories of best practices. We then describe our center's scientific workflow (from hypothesis formulation to result reporting) and detail how each element of this workflow maps onto these four categories. We also provide specific examples of practices or tools that support this mapping process. Finally, we offer a roadmap for the stepwise adoption of these practices, providing recommendations of why and what to do as well as a summary of cost-benefit tradeoffs for each step of the transition.

The past, present and future of neuroscience data sharing: a perspective on the state of practices and infrastructure for FAIR

“…Across all stages of the data lifecycle, the management strategy puts in place processes so that data can be found, accessed, combined when necessary, and reused. By paying attention to FAIR in the laboratory throughout the life cycle, benefits start to accrue to the data creator, the laboratory, PI, and collaborators well before data flows out to the wider scientific community (Bush et al, 2022;Dempsey et al, 2022).…”

“…We are starting to see neuroscience researchers sharing their experiences with developing and utilizing lab-centric data management systems. They range from tightly integrated digital infrastructures (Bush et al, 2022;Dempsey et al, 2022) to a set of…”

“…BIDS, for example, has been estimated to have been used to organize over 100,000 datasets containing millions of images, indicating significant uptake by the research community (Poldrack et al, 2023). In a recent paper that outlined a neuroimaging center's implementation of BIDS, Bush et al (2022) stated: "Learning the BIDS specification, implementing software pipelines to map the data, and validating that the resultant mappings met the BIDS standard consumed many months of effort across multiple imaging center team members… The benefits of mapping our data to BIDS, however, far exceed the costs. " (Bush et al, 2022).…”

“…In a recent paper that outlined a neuroimaging center's implementation of BIDS, Bush et al (2022) stated: "Learning the BIDS specification, implementing software pipelines to map the data, and validating that the resultant mappings met the BIDS standard consumed many months of effort across multiple imaging center team members… The benefits of mapping our data to BIDS, however, far exceed the costs. " (Bush et al, 2022). These benefits included access to BID-APPS, a set of containerized analysis tools and pipelines that run on validated BIDS data, as well as improved code sharing within the lab and with colleagues, as well as a reduced barrier to publishing the data in OpenNeuro.…”

“…Bush et al (2022) Consider creating a lab-wide data dictionary where all variables used across experiments are clearly definedBush et al (2022);Fouad et al (2023) Reusable Documentation Create a "Read me" file for each dataset where notes can be captured and helpful information provided for reuse of the data Community Standards All files should be collected and stored in well supported open formats ideally to ensure long term availability. Adopt community standards within the lab where possible; a good place to identify relevant standards is to look at repositories where the data may end up.…”