Averting the next credibility crisis in Psychological Science: Within-person methods for personalized diagnostic and intervention

Abstract

Personalizing assessments, predictions, and treatments of individuals is currently a defining trend in psychological research and applied fields, including personalized learning, personalized medicine, and personalized advertisement. For instance, the recent pandemic has reminded parents and educators of how challenging yet crucial it is to get the right learning task to the right student at the right time. Increasingly, psychologists and social scientists are realizing that the between-person methods that we have long been using in the hope to describe, predict, and treat individuals may fail to live up to these tasks (e.g., Molenaar, 2004). Consequently, there is a risk of a credibility loss, possibly similar to the one seen during the replicability crisis (Ioannides, 2005), because we have only started to understand how many of the conclusions that we tend to draw based on between-person methods misunderstand what these methods can tell us and what they cannot. An imminent methodological revolution will likely change even very established psychological theories (Barbot et al., 2020). Fortunately, methodological solutions for personalized descriptions and predictions, such as many within-person analyses, are available and rapidly being developed, although they are not yet embraced in all areas of Psychology, and some come with their own limitations. This article first discusses the extent of the theory-method gap between theories about within-person patterns versus methods examining only between-person patterns in Psychology, and the potential loss of trust that might follow from these limitations of the commonly used between-person methods. Second, this article addresses advantages and limitations of available within-person methods. Third, this article discusses how within-person analytical methods may help improving the individual descriptions and predictions that are needed in many applied fields aiming for tailored individual solutions, including personalized learning with educational technology and personalized medicine.

What Correlations Mean for Individual People: A Tutorial for Researchers, Students and the Public

“…When using it, I suspect that some readers will find the answers more sobering than they expected. Even when disregarding other issues that complicate using correlations to say something about individuals, such as the ecological fallacy (e.g., Simpson's paradox; Kievit et al, 2013;Moeller, 2021), typical correlations reported in psychological research are so weak that even seemingly intuitive conclusions about individuals are not only uncertain but often more likely to be wrong than right.…”