Working with psychology journal editors to correct problems in the scientific literature.

Abstract

Psychology faces a replicability crisis in which its credibility as a science is in question. A recommended prerequisite to exploring replicability is to first explore reproducibility by carefully reexamining existing studies for any internal problems prior to attempting to externally replicate them. However, a frequent complaint from people attempting to get errors found in existing studies corrected within the psychological literature is some journal editors' reticence to publish critical comments, and especially to issue corrections and retractions, on articles found to be problematic. These editors seem to view criticisms as negative and destructive, rather than positive and constructive, and can obstruct important corrections needed for psychological science to be more adaptive. As powerful gatekeepers, editors play a crucial role in perpetuating a dysfunctional scientific culture that undermines psychology's viability. However, effective advocacy can successfully challenge editors' resistance to publishing corrections and taking other responsible actions, as well as confront pushback from authors' attempts to shield their criticized articles from scrutiny. We present reproducibility efforts that resulted in a debunking article, several critical commentaries, and a letter to the editor being published, as well as several published corrections and an expression of concern, but no retractions. We discuss these examples in terms of confronting what we see as a dysfunctional culture that often protects the status quo by opposing corrections, which is analyzed as a perverse incentive. We also offer suggestions for changing this dysfunctional culture into one that is more in accord with science as an adaptive, self-correcting system. Public Significance StatementPsychological science undergirds all of the field's applications, yet it suffers from widespread replicability and reproducibility failures; even many of its foundational studies have been shown to be flawed. Change is needed to restore psychology's viability, which requires addressing its dysfunctional scientific culture, including those aspects that discourage journal editors from publishing corrections to the scientific literature. Several cases of editorial resistance to publishing reports debunking findings that could not be reproduced are examined, and recommendations are offered to encourage editors, and others, to embrace, rather than obstruct, self-correcting psychological science.

Where Are the Self-Correcting Mechanisms in Science?

“…In theory, journals themselves provide one potential avenue for such post-publication corrections, via published commentaries, critiques, and corrections. However, anecdotal reports suggest that the bar for publishing reports of errors in a scientific paper can be very high (Heathers, 2015; Pickett, 2020; Friedman et al, in press; Goldacre et al, 2019).…”

“…Journals and universities have strong disincentives to admit that their own research is faulty, and the Office of Research Integrity is woefully under-resourced (Retraction Watch, 2014). Heathers (2015), Pickett (2020), and Friedman et al, (in press) have provided first-person accounts of the difficulty of bringing errors or misconduct to light through such formal channels as journals or university disciplinary committees. Clearly, these safeguards often are not effective at ensuring that self-correction is prioritized.…”

“…The adoption of preregistration can facilitate accumulating such a track record. One reason, we believe, that the role of theory in social sciences is hotly contested (Meehl, 1990; Yarkoni & Westfall, 2017; Fried, 2020; Cummins, 2000; van Rooij & Baggio, 2020) is because we have so little unbiased data, and little agreement, on where and when theories make good predictions. It may be that some fields already contain quantitative theories that have been making successful predictions, but the absence of preregistration as a practice has made this hard to recognize.…”

Introduction to special section on human-animal studies: Humanistic psychology contributions of Team Woof.

“…Their argument worked, and Bloom (2011) not only completed her dissertation but, when published as an article (Bloom & Friedman, 2013) and later a chapter (Bloom & Friedman, 2014), it went viral, being widely covered by national (Boyle, 2013) and even international (Lusher, 2013) media. Due to this unexpected recognition, Bloom and Friedman informally created “Team Woof” to pursue this line of research, which included obtaining grant funding used to successfully replicate and extend the initial study (Bloom et al, 2021), something especially pleasing in light of psychology’s current crisis in which many headline-grabbing findings fail to replicate (e.g., Friedman et al, 2020). Melissa Trevathan-Minnis later joined Team Woof, and a number of other collaborations have ensued, including several international ones.…”

Building a cumulative psychological science.

“…Journal editors are largely responsible for the publication process. Refusal or hesitation by journal editors to publish critical comments on flawed articles is the focus of Friedman, MacDonald, and Coyne (2020, pp. 342–348).…”