Putting the Self in Self-Correction: Findings From the Loss-of-Confidence Project

Abstract

Science is often perceived to be a self-correcting enterprise. In principle, the assessment of scientific claims is supposed to proceed in a cumulative fashion, with the reigning theories of the day progressively approximating truth more accurately over time. In practice, however, cumulative self-correction tends to proceed less efficiently than one might naively suppose. Far from evaluating new evidence dispassionately and infallibly, individual scientists often cling stubbornly to prior findings. Here we explore the dynamics of scientific self-correction at an individual rather than collective level. In 13 written statements, researchers from diverse branches of psychology share why and how they have lost confidence in one of their own published findings. We qualitatively characterize these disclosures and explore their implications. A cross-disciplinary survey suggests that such loss-of-confidence sentiments are surprisingly common among members of the broader scientific population yet rarely become part of the public record. We argue that removing barriers to self-correction at the individual level is imperative if the scientific community as a whole is to achieve the ideal of efficient self-correction.

No harm in being self-corrective: Self-criticism and reform intentions increase researchers’ epistemic trustworthiness and credibility in the eyes of the public

“…Our findings suggest that researchers’ hesitation toward self-correction (e.g. Fetterman and Sassenberg, 2015 ; Frewer et al, 2003 ; Rohrer et al, 2021 ; van der Bles et al, 2020 ) seems unwarranted: there is no harm in openly admitting doubts and regarding one’s findings as preliminary or in intending to reform one’s work routines. On the contrary, researchers who portray their findings as fixed and definite and who are unwilling to implement reforms are perceived as less trustworthy and less credible by laypeople.…”

“…However, empirical findings suggest that the ever-doubtful and self-correcting scientist is an ideal and that, when it comes to admitting flaws and failures, researchers behave just as “normal” people do: they hesitate to do so for the sake of protecting their self-image and their reputation ( Bishop, 2018 ; Fetterman and Sassenberg, 2015 ; Rohrer et al, 2021 ; van der Bles et al, 2020 ). Thus, the question is whether these concerns are justified.…”

Recommendations from the Reducing the Inadvertent Spread of Retracted Science: Shaping a Research and Implementation Agenda Project

“…Editors at The Lancet and The 5 Such culture change seems to be starting in some fields: the "loss-of-confidence" project focuses on self-correction efforts (including but not limited to retraction) from empirical reports of novel findings in psychology; they say "even the 13 submissions we ultimately received could be considered a clear success and a testament to the current introspection and self-critical climate in psychology." (Rohrer et al, 2021). 6 This is referred to as "Retractedandrepublished" in the PubMed XML help (U.S. National Library of Medicine, 2021).…”

A Call for Greater Modesty in Psychology and Cognitive Neuroscience

“…And individuals are certainly part of it and should therefore shoulder some of the responsibility, as has been argued previously (Ritchie, 2020;Yarkoni, 2018). This could take the form of more modest initial claims, as well as more routine engagement with public self-correction of previously published work (Rohrer et al, 2021). A hopeful note, which may provide some motivation for a modest approach, would be that real change can only occur when many small steps are taken at many different levels.…”