Reproducibility failures are essential to scientific inquiry

Abstract

Current fears of a "reproducibility crisis" have led researchers, sources of scientific funding, and the public to question both the efficacy and trustworthiness of science (1, 2). Suggested policy changes have been focused on statistical problems, such as p-hacking, and issues of experimental design and execution (3, 4). However, "reproducibility" is a broad concept that includes a number of issues (5) (see also www.pnas. org/improving_reproducibility). Furthermore, reproducibility failures occur even in fields such as mathematics or computer science that do not have statistical problems or issues with experimental design. Most importantly, these proposed policy changes ignore a core feature of the process of scientific inquiry that

The limits of replicability

“…Some authors have gone further than this and claimed that failure in general is not a sign of crisis but just part of normal science (see, e.g., Firestein 2015;Redish et al 2018). Whilst these views share a rejection of general crisis talk with the new localism, the latter represents a middle position in this often-heated debate, as its main point is not to question crisis talk per se.…”

Rinse and Repeat: Understanding the Value of Replication across Different Ways of Knowing

“…Whether or not the label 'crisis' is legitimate is a reasonable question and qualifications differ remarkably [2][3][4][5][6][7]. However, there obviously is momentum generated through the rise of the reproducibility movement and the new socio-technical (or methodological) infrastructure that is emerging in science to avoid irreplicability.…”

Rinse and repeat: understanding the value of replication across different ways of knowing

“…Whether or not the label 'crisis' is legitimate is a reasonable question and qualifications differ remarkably [2][3][4][5][6][7]. However, there obviously is momentum generated through the rise of the Openness ambitions are growing and policy makers are rolling out new policies to encourage openness across entire academic communities, regardless of discipline, epistemology, object of study, digital and financial capabilities and more.…”