Estimating the reproducibility of psychological science

Abstract

52% Yes, a signiicant crisis 3% No, there is no crisis 7% Don't know 38% Yes, a slight crisis 38% Yes, a slight crisis 1,576 RESEARCHERS SURVEYED M ore than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments. Those are some of the telling figures that emerged from Nature's survey of 1,576 researchers who took a brief online questionnaire on reproducibility in research. The data reveal sometimes-contradictory attitudes towards reproduc-ibility. Although 52% of those surveyed agree that there is a significant 'crisis' of reproducibility, less than 31% think that failure to reproduce published results means that the result is probably wrong, and most say that they still trust the published literature. Data on how much of the scientific literature is reproducible are rare and generally bleak. The best-known analyses, from psychology 1 and cancer biology 2 , found rates of around 40% and 10%, respectively. Our survey respondents were more optimistic: 73% said that they think that at least half of the papers in their field can be trusted, with physicists and chemists generally showing the most confidence. The results capture a confusing snapshot of attitudes around these issues, says Arturo Casadevall, a microbiologist at the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland. "At the current time there is no consensus on what reproducibility is or should be. " But just recognizing that is a step forward, he says. "The next step may be identifying what is the problem and to get a consensus. "

Differential patterns of age‐related cortical and subcortical functional connectivity in 6‐to‐10 year old children: A connectome‐wide association study

“…This study also effectively used “brain‐wide” visualizations to display large amounts of connectomic information, namely in the connectograms and worm plots. In addition, we present both novel findings as well as replication of observations from earlier studies, the latter being important in neuroscience, which is a field plagued by many underpowered studies that do not replicate (Nichols et al., 2017; Open Science Collaboration, 2015). …”

When null hypothesis significance testing is unsuitable for research: a reassessment

“…There is increasing discontent that many areas of psychological science, cognitive neuroscience, and biomedical research (Ioannidis, 2005;Ioannidis et al, 2014) are in a crisis of producing too many false positive non-replicable results (Begley and Ellis, 2012;Aarts et al, 2015). This wastes research funding, erodes credibility and slows down scientific progress.…”

Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape

“…However, they are also hard to implement and even harder to reproduce in other contexts. Recently, there has been an increasing focus on reproducibility in psychology (Open Science Collaboration, 2015) and worries about claims that only certain researchers have the right ‘flair’ to replicate studies (Baumeister, 2016). Confederate studies in particular may be susceptible to such effects (Doyen, Klein, Pichon, & Cleeremans, 2012), or participants may be behave differently with confederates (Kuhlen & Brennan, 2013).…”