Do Neuroscience Journals Accept Replications? A Survey of Literature

Yeung, Andy Wai Kan

doi:10.3389/fnhum.2017.00468

Cited by 33 publications

(27 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All of these may influence the quality of an article. Publishing replication studies regardless of statistical significance may help readers better comprehend the data quality ( Yeung, 2017 ). Meanwhile, conducting meta-analysis of functional neuroimaging data can also establish consensus on the locations of brain activation to confirm or refute hypothesis ( Wager et al, 2007 ; Zmigrod et al, 2016 ; Yeung et al, 2017b , d , 2018 ).…”

Section: Discussionmentioning

confidence: 99%

An Updated Survey on Statistical Thresholding and Sample Size of fMRI Studies

Yeung

2018

Front. Hum. Neurosci.

Self Cite

View full text Add to dashboard Cite

Background: Since the early 2010s, the neuroimaging field has paid more attention to the issue of false positives. Several journals have issued guidelines regarding statistical thresholds. Three papers have reported the statistical analysis of the thresholds used in fMRI literature, but they were published at least 3 years ago and surveyed papers published during 2007–2012. This study revisited this topic to evaluate the changes in this field. Methods: The PubMed database was searched to identify the task-based (not resting-state) fMRI papers published in 2017 and record their sample sizes, inferential methods (e.g., voxelwise or clusterwise), theoretical methods (e.g., parametric or non-parametric), significance level, cluster-defining primary threshold (CDT), volume of analysis (whole brain or region of interest) and software used. Results: The majority (95.6%) of the 388 analyzed articles reported statistics corrected for multiple comparisons. A large proportion (69.6%) of the 388 articles reported main results by clusterwise inference. The analyzed articles mostly used software Statistical Parametric Mapping (SPM), Analysis of Functional NeuroImages (AFNI), or FMRIB Software Library (FSL) to conduct statistical analysis. There were 70.9%, 37.6%, and 23.1% of SPM, AFNI, and FSL studies, respectively, that used a CDT of p ≤ 0.001. The statistical sample size across the articles ranged between 7 and 1,299 with a median of 33. Sample size did not significantly correlate with the level of statistical threshold. Conclusion: There were still around 53% (142/270) studies using clusterwise inference that chose a more liberal CDT than p = 0.001 ( n = 121) or did not report their CDT ( n = 21), down from around 61% reported by Woo et al. (2014) . For FSL studies, it seemed that the CDT practice had no improvement since the survey by Woo et al. (2014) . A few studies chose unconventional CDT such as p = 0.0125 or 0.004. Such practice might create an impression that the threshold alterations were attempted to show “desired” clusters. The median sample size used in the analyzed articles was similar to those reported in previous surveys. In conclusion, there seemed to be no change in the statistical practice compared to the early 2010s.

show abstract

Section: Discussionmentioning

confidence: 99%

An Updated Survey on Statistical Thresholding and Sample Size of fMRI Studies

Yeung

2018

Front. Hum. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, there seems to be a substantial bias toward publishing positive results, which might technically inflate false positives rates (Eklund, Nichols, & Knutsson, ) and may contribute to the possibility of a large amount of spurious findings. In addition, it has been argued that the appeal for publishing new positive findings frequently obscures the truth about underlying mechanisms behind such disorders, even with limited replication studies (Fletcher & Grafton, ; Tahmasian et al, , ; Yeung, ).…”

Section: Introductionmentioning

confidence: 99%

Practical recommendations to conduct a neuroimaging meta‐analysis for neuropsychiatric disorders

Tahmasian

Sepehry

Samea

et al. 2019

Human Brain Mapping

137

View full text Add to dashboard Cite

Over the past decades, neuroimaging has become widely used to investigate structural and functional brain abnormality in neuropsychiatric disorders. The results of individual neuroimaging studies, however, are frequently inconsistent due to small and heterogeneous samples, analytical flexibility, and publication bias toward positive findings. To consolidate the emergent findings toward clinically useful insight, meta‐analyses have been developed to integrate the results of studies and identify areas that are consistently involved in pathophysiology of particular neuropsychiatric disorders. However, it should be considered that the results of meta‐analyses could also be divergent due to heterogeneity in search strategy, selection criteria, imaging modalities, behavioral tasks, number of experiments, data organization methods, and statistical analysis with different multiple comparison thresholds. Following an introduction to the problem and the concepts of quantitative summaries of neuroimaging findings, we propose practical recommendations for clinicians and researchers for conducting transparent and methodologically sound neuroimaging meta‐analyses. This should help to consolidate the search for convergent regional brain abnormality in neuropsychiatric disorders.

show abstract

“…An overview of the existing simulation tools, their features and strengths, as well as a discussion on future developments are presented in recent publications (Brette et al, 2007 ; Crook et al, 2013 ; McDougal et al, 2016 ). In addition, journals rarely explicitly state that they accept replicability and reproducibility studies (Yeung, 2017 ). However, the ReScience initiative was started to encourage researchers to reimplement published models and reproduce the research results (Rougier et al, 2017 ).…”

Section: Introductionmentioning

confidence: 99%

Challenges in Reproducibility, Replicability, and Comparability of Computational Models and Tools for Neuronal and Glial Networks, Cells, and Subcellular Structures

Manninen

Acimovic

Havela

et al. 2018

Front. Neuroinform.

View full text Add to dashboard Cite

The possibility to replicate and reproduce published research results is one of the biggest challenges in all areas of science. In computational neuroscience, there are thousands of models available. However, it is rarely possible to reimplement the models based on the information in the original publication, let alone rerun the models just because the model implementations have not been made publicly available. We evaluate and discuss the comparability of a versatile choice of simulation tools: tools for biochemical reactions and spiking neuronal networks, and relatively new tools for growth in cell cultures. The replicability and reproducibility issues are considered for computational models that are equally diverse, including the models for intracellular signal transduction of neurons and glial cells, in addition to single glial cells, neuron-glia interactions, and selected examples of spiking neuronal networks. We also address the comparability of the simulation results with one another to comprehend if the studied models can be used to answer similar research questions. In addition to presenting the challenges in reproducibility and replicability of published results in computational neuroscience, we highlight the need for developing recommendations and good practices for publishing simulation tools and computational models. Model validation and flexible model description must be an integral part of the tool used to simulate and develop computational models. Constant improvement on experimental techniques and recording protocols leads to increasing knowledge about the biophysical mechanisms in neural systems. This poses new challenges for computational neuroscience: extended or completely new computational methods and models may be required. Careful evaluation and categorization of the existing models and tools provide a foundation for these future needs, for constructing multiscale models or extending the models to incorporate additional or more detailed biophysical mechanisms. Improving the quality of publications in computational neuroscience, enabling progressive building of advanced computational models and tools, can be achieved only through adopting publishing standards which underline replicability and reproducibility of research results.

show abstract

Do Neuroscience Journals Accept Replications? A Survey of Literature

Cited by 33 publications

References 20 publications

An Updated Survey on Statistical Thresholding and Sample Size of fMRI Studies

An Updated Survey on Statistical Thresholding and Sample Size of fMRI Studies

Practical recommendations to conduct a neuroimaging meta‐analysis for neuropsychiatric disorders

Challenges in Reproducibility, Replicability, and Comparability of Computational Models and Tools for Neuronal and Glial Networks, Cells, and Subcellular Structures

Contact Info

Product

Resources

About