Philosophy of science and the replicability crisis

Abstract

Replicability is widely taken to ground the epistemic authority of science. However, in recent years, important published findings in the social, behavioral, and biomedical sciences have failed to replicate, suggesting that these fields are facing a “replicability crisis.” For philosophers, the crisis should not be taken as bad news but as an opportunity to do work on several fronts, including conceptual analysis, history and philosophy of science, research ethics, and social epistemology. This article introduces philosophers to these discussions. First, I discuss precedents and evidence for the crisis. Second, I discuss methodological, statistical, and social‐structural factors that have contributed to the crisis. Third, I focus on the philosophical issues raised by the crisis. Finally, I discuss several proposals for solutions and highlight the gaps that philosophers could focus on.

Why Most Research Findings About Psi Are False: The Replicability Crisis, the Psi Paradox and the Myth of Sisyphus

“…The replicability crisis has been illustrated by the results of nine experiments conducted by Bem (2011) and reported in the Journal of Personality and Social Psychology as supporting the existence of precognition. As Romero (2019) explains "although the finding persuaded very few scientists, the controversy engendered mistrust in the ways psychologists conduct their experiments because Bem used procedures and statistical tools that many social psychologists use" (p. 3). Indeed, if Bem was able to demonstrate the existence of precognition -and given that precognition cannot exist for a lot of psychologists (Reber and Alcock, 2020) -did he show unwittingly that something was profoundly wrong in the way experiments are conducted in the field of psychology (Wiggins and Chrisopherson, 2019)?…”

Double trouble? The communication dimension of the reproducibility crisis in experimental psychology and neuroscience

“…However, out of those 1.07%, only 47.1% were performed by independent investigators and 14% were direct rather than conceptual replications. When coupled with the publication bias, a large number of underpowered conceptual replication studies can, in fact, create an illusion of robustness (Romero 2019). Given these figures, it is nearly impossible to arrive at a credible estimate of reproducibility based on prior replication attempts.…”

Replications in Comparative Psychology

“…Roughly, a direct replication tests the same hypothesis using very similar methods (treatment, setup, measurement, etc.) but different subject groups, while a conceptual replication tests the same hypothesis using different methods (Romero 2019).…”

“…In the last decade, efforts to replicate research have increased in a wide range of sciences, including medicine, psychology, and computer science (Bohannon 2014). This replication initiative comes in part in response to a series of failed replication attempts and the growing awareness that certain standard scientific practices need to be evaluated and reformed (Romero 2019). Thus far, comparative psychology has not been centrally involved in this so-called 'replication crisis' (but see Farrar & Ostojić 2019;Farrar et al 2020).…”