Survey on Open Science Practices in Functional Neuroimaging

Paret, Christian; Unverhau, Nike; Feingold, Franklin; Poldrack, Russell A.; Stirner, Madita; Schmahl, Christian; Sicorello, Maurizio

doi:10.1016/j.neuroimage.2022.119306

Cited by 36 publications

(25 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, the challenge with relying on exploration and general predictions has a long history in the behavioral sciences representing important challenges to falsifiability and theory building. 50,51 Understandably, the emergence of novel methods is commonly accompanied by tentative hypotheses (for brain imaging in TBI see 52 and more generally see 53,54 , but even with the growth and maturity of the scientific communities examined here, we find little associated change in the strength of the hypotheses.…”

Section: Tracking Influential Hypothesesmentioning

confidence: 72%

Establishing ground truth in the clinical neurosciences: if replication is the answer, then what are the questions?

Priestley¹,

Staph²,

Koneru³

et al. 2022

Preprint

View full text Add to dashboard Cite

The replication crisis poses important challenges to modern science. Central to this challenge is re-establishing ground truths, or the most fundamental theories that serve as the bedrock to a scientific community. However, the goal to identify hypotheses with the greatest support is nontrivial given the current unprecedented rate of scientific publishing. In this era of high-volume science, the goal of this study is to sample from one research community within clinical neuroscience (traumatic brain injury; TBI) and track major trends that have shaped this literature over the past 50 years. To do so, we first conduct decade-wise (1980-2019) network analysis to examine the scientific communities that shape this literature. To establish the robustness of our network findings, we utilized searches from separate search engines (Web of Science; Semantic Scholar). As a second goal, we sought to determine the most highly cited hypotheses influencing the literature in each decade. In a third goal, we then searched for any papers referring to “replication” or efforts to reproduce findings within our >50,000 paper dataset. From this search, 550 papers were analyzed to determine the frequency and nature of formal replication studies over time. Finally, to maximize transparency, we provide a detailed procedure for the creation and analysis of our dataset, including a discussion of each of our major decision points, to facilitate similar efforts in other areas of neuroscience. We found that the unparalleled rate of scientific publishing within the TBI literature combined with the scarcity of clear hypotheses in individual publications are a challenge to both evaluating accepted findings and determining future directions. Additionally, while the conversation about reproducibility has increased over the past decade, the rate of published replication studies continues to be a negligible proportion of the research. Meta-science and computational methods offer the critical opportunity to assess the state of the science and illuminate pathways forward, but ultimately there is structural change needed in the TBI literature and perhapsothers.

show abstract

Section: Tracking Influential Hypothesesmentioning

confidence: 72%

Establishing ground truth in the clinical neurosciences: if replication is the answer, then what are the questions?

Priestley¹,

Staph²,

Koneru³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…search of the 'best results (Lindquist, 2020;Paret et al, 2022). In the absence of appropriate preregistration and reporting standards, this can cause systematic inflation of type 1 error and incorrect inferences (Flournoy et al, 2020;Verschuere et al, 2021).…”

Section: Discussionmentioning

confidence: 99%

No Impaired Integration in Psychopathy: Evidence from an Illusory Conjunction Paradigm

Gunschera¹,

Verschuere²,

Murphy³

et al. 2022

Preprint

View full text Add to dashboard Cite

Progress in psychopathy research has been hampered by ongoing contention about its fundamental cause. The Impaired Integration theory of psychopathy provides an attention-based account of information integration abnormalities. We set out to evaluate the suggested mechanism via an innovative application of the well-established illusory conjunction paradigm (Treisman & Gelade, 1980). Two hundred participants were recruited by utilising a psychopathic-trait-maximisation technique, sampling individuals from an ex-prisoner and a population sample. We found no evidence for differences in illusory conjunctions suggesting that integration of cues deficits in psychopathic individuals (BF10 = 0.156), and the absence of a relationship between psychopathic traits and illusory conjunctions remained when accounting for confounding variables. These findings question the mechanism proposed by the Impaired Integration theory and pave the way for future research to advance our understanding of psychopathic trait aetiology by assessing specific and falsifiable mechanisms thought to bring about the observed cognitive and behavioural deficits.

show abstract

“…While there is growing recognition of the imperative of implementing these best practices there exits, in our opinion, a clear absence of how procedurally to accomplish this goal (Paret et al, 2022 ). This potential barrier to widespread adoption of open science practices is echoed for other fields (Kalandadze and Hart, 2022 ).…”

Section: Introductionmentioning

confidence: 99%

Lessons learned: A neuroimaging research center's transition to open and reproducible science

2022

View full text Add to dashboard Cite

Human functional neuroimaging has evolved dramatically in recent years, driven by increased technical complexity and emerging evidence that functional neuroimaging findings are not generally reproducible. In response to these trends, neuroimaging scientists have developed principles, practices, and tools to both manage this complexity as well as to enhance the rigor and reproducibility of neuroimaging science. We group these best practices under four categories: experiment pre-registration, FAIR data principles, reproducible neuroimaging analyses, and open science. While there is growing recognition of the need to implement these best practices there exists little practical guidance of how to accomplish this goal. In this work, we describe lessons learned from efforts to adopt these best practices within the Brain Imaging Research Center at the University of Arkansas for Medical Sciences over 4 years (July 2018–May 2022). We provide a brief summary of the four categories of best practices. We then describe our center's scientific workflow (from hypothesis formulation to result reporting) and detail how each element of this workflow maps onto these four categories. We also provide specific examples of practices or tools that support this mapping process. Finally, we offer a roadmap for the stepwise adoption of these practices, providing recommendations of why and what to do as well as a summary of cost-benefit tradeoffs for each step of the transition.

show abstract

Survey on Open Science Practices in Functional Neuroimaging

Cited by 36 publications

References 42 publications

Establishing ground truth in the clinical neurosciences: if replication is the answer, then what are the questions?

Establishing ground truth in the clinical neurosciences: if replication is the answer, then what are the questions?

No Impaired Integration in Psychopathy: Evidence from an Illusory Conjunction Paradigm

Lessons learned: A neuroimaging research center's transition to open and reproducible science

Contact Info

Product

Resources

About