“So you confirmed, replicated and emptied your file-drawer… now what?” A Structural Realist’s Guide to Theory Evaluation in Psychological Science

Abstract

The papers in the special issue of PPS on the origins and possible resolution of the crisis of confidence in psychological science hardly mention the word theory. We argue that all the excellent suggestions to reform the discipline will be in vain if it does not begin a discussion about the way it evaluates and revises its theories. We diagnose theory evaluation as afflicted with three serious symptoms and provide exemplary examples from the scientific record of psychology to illustrate them. The first symptom is the interpretation fallacy, of which the very critical commentaries received by proponents of the complex systems approach about their efforts to study psychological phenomena, represent an exemplary case (e.g. Wagenmakers, van der Maas, & Farrell, 2012). The second is the empiricist’s blind-spot. We will discuss it in the context of the plethora of theories that have been posited by the psychological and life sciences about the ætiology of developmental dyslexia. We especially discuss the apparent inability of the huge empirical record of high quality behavioral, neuroscientific and genetic data to resolve any theoretical dispute (e.g. Ramus & Szenkovits, 2008). The third symptom is the Columbus delusion, of which the recent publication “There is nothing so theoretical as a good method” (Greenwald, 2012) is an excellent example. These symptoms are all related and pertain to the different roles ontology plays in scientific theorizing. Our critical diagnosis is not the end goal of this paper however, we respond to the call to device methods for strong inference (Fiedler, Kutzner, & Krueger, 2012) and provide a practical, but formal approach to theory evaluation known as Structural Realism (cf. Worrall, 1989). In three parts, our Structural Realist’s guide to theorizing about the universe will elaborate on how to deal with theoretical diversity and in the process cure psychological science of its ailment, ensuring it will remain the prime discipline for the scientific study of psychological phenomena in the future.

Going round in squares: Theory-based measurement requires a theory of measurement

“…The first claim I will defend in this commentary is that even if a revolution happens in psychological theorizing that will finally provide us with substantive formal theories, without a formalization of the process of measurement of psychological phenomena, the discipline will be back to square one—inferring the best-fitting parameters of statistical (latent variable) models from noisy data but still lacking a clear notion of how to incorporate the measurement context and the act of measurement into the description of the psychological phenomenon (Hasselman et al, 2019). The second claim I will defend concerns the following statement: “Our critique of LVM [latent variable modeling] is independent of the specific assumptions made in latent variable models, like quantitative structure, ergodicity, local independence, and so forth” (Borgstede & Eggert, 2023, p. 127).…”

How children make sense of the world: A perceptual learning account

“…In their study, they infer that the empirically observable 'direct gaze' operates as an 'ostensive cue' , but they seem to describe ostensive cues as a part of the empirical reality that can be observed by the infants in their study. This type of fallacy seems omnipresent within the cognitive view (see Hasselman, Seevinck, & Cox, 2019) which turns an inference such as the existence of an internal mental representation into an invisible yet material ghost that taunts the scientific empirical endeavour of social sciences. On logical grounds, the mental realm cannot be refuted from the ecological view based on the argument known as incommensurability (Kuhn, 1962).…”