Metascience Is Not Enough – A Plea for Psychological Humanities in the Wake of the Replication Crisis

Abstract

The replication crisis led to the rise of metascience as a possible solution. In this article, we examine central metascientific premises and argue that attempts to solve the replication crisis in psychology will benefit from a tighter integration of approaches from the psychological humanities. The first part of our article identifies central epistemic merits that metascientific endeavors can contribute to psychology. However, we argue secondly against the widespread claim that metascience is the only way to deal with the replication crisis in psychology and point to major epistemic problems: the one-sided notion of a singular scientific method, the homogenizing view of psychology, and the exclusion of practices of theorizing. As a possible compensation for such shortcomings, we introduce, third, the reflective and pluralistic approach of psychological humanities. In so doing, we show how psychological humanities can serve as an important complement to the objective of improving psychological research. Psychological humanities contribute to a more precise determination of validity, to ethical considerations, and a better understanding of psychology’s objects in regard to replication. Accordingly, we argue for the integration of psychological humanities into both metascience and psychology to provide a better basis for addressing epistemic and ethical questions.

Rating scales institutionalise a network of logical errors and conceptual problems in research practices: A rigorous analysis showing ways to tackle psychology’s crises

“…Yet psychology’s continued crises about its findings, theories and research practices testify to fundamental problems still unsolved. Current initiatives to tackle these problems (e.g., open and ‘meta’ science; Malich and Rehmann-Sutter, 2022 ) are all targeted at improving data analysis and interpretation (e.g., response coding and transformation; construct operationalisation and validity; statistical tests; Flake and Fried, 2020 ; Hardwicke et al, 2022 ). But rating ‘scales’ remained largely unchallenged.…”

Questionable Metascience Practices

“…As several commentators have observed, contemporary metascientists tend to be concerned with how bias and motivated reasoning influence scientists' methods, analyses, and interpretations (Field & Derksen, 2021 2022) has suggested that metascientists may be biased towards explaining the replication crisis in terms of researcher bias because they are overrepresented by psychologists (Moody et al, 2022; see also Flis, 2019;Malich & Rehmann-Sutter, 2022), who tend to be familiar with cognitive and motivational biases (i.e., a type of availability heuristic bias). Consistent with Morawski's interpretation, it is interesting to note that psychologists' metabias may also explain their emphasis on researcher bias during the 1960s-1970s crisis of confidence in social psychology (Peterson & Panofsky, 2021, p. 600;Rosnow, 1983).…”

Responsible assessment of what research? Beware of epistemic diversity!

“…Similarly, the proposal from the authors seems to be targeting research where writing has the purpose of reporting, but in rather theoretical, conceptual and hermeneutical work writing itself is largely the research product which therefore has to be judged based on different criteria because the act of writing also has different purposes (Penders et al, 2020). Hence, the authors neglect the plurality of approaches to knowledge production (ways of knowing) within psychological science and assume a fantastic uniform discipline that never existed (see e.g., Malich & Rehmann-Sutter, 2022;Koch, 1993). Furthermore, there seems to be some 4 nuance missing when it comes to claims concerning the appropriateness and applicability of some open science practices across the scientific landscape in psychology (see e.g., Guzzo et al, 2023).…”