Beneath the surface: Unearthing within-person variability and mean relations with Bayesian mixed models.

Abstract

Mixed-effects models are becoming common in psychological science. Although they have many desirable features, there is still untapped potential. It is customary to view homogeneous variance as an assumption to satisfy. We argue to move beyond that perspective, and to view modeling within-person variance as an opportunity to gain a richer understanding of psychological processes. The technique to do so is based on the mixed-effects location scale model that can simultaneously estimate mixed-effects sub-models to both the mean (location) and within-person variance (scale). We develop a framework that goes beyond assessing the sub-models in isolation of one another and introduce a novel Bayesian hypothesis test for mean-variance correlations in the distribution of random effects. We first present a motivating example, which makes clear how the model can characterize mean-variance relations. We then apply the method to reaction times gathered from two cognitive inhibition tasks. We find there are more individual differences in the within-person variance than the mean structure, as well as a complex web of structural mean-variance relations. This stands in contrast to the dominant view of within-person variance (i.e., "noise"). The results also point towards paradoxical within-person, as opposed to between-person, effects: several people had slower and less variable incongruent responses. This contradicts the typical pattern, wherein larger means tend to be associated with more variability. We conclude with future directions, spanning from methodological to theoretical inquires, that can be answered with the presented methodology.

Understanding the Error in Psychopathology: Notable Intraindividual Differences in Neural Variability of Performance Monitoring

“…Another consequence of the between-person differences in intraindividual variability is that group-based estimates of psychometric internal consistency risk masking low reliability in some participants (44,56,77,81). The assumption of fixed within-person variance in groupbased estimates can be relaxed so persons have their own estimate of "error variance" for calculating person-specific reliability (44,56).…”

When (Not) to Rely on the Reliable Change Index: A Critical Appraisal and Alternatives to Consider in Clinical Psychology

“…It would be ideal to have within-person estimates of variability in the RCI (Donaldson, 2008;Hays & Peipert, 2021). More advanced methods, such as item-response theory methods , account for additional sources of variability in measurement error, and models that estimate person-specific measurement error (e.g., Williams et al, 2021) may become widely used in the future. This assumption is common to a range of testing applications, even though it may not be optimal.…”

Statements About the Pervasiveness of Behavior Require Data About the Pervasiveness of Behavior

“…Though Fisher et al's (2018) study is a very positive sign of growing awareness of this issue, the problem is not a new one, and has been raised repeatedly by others to little apparent effect on mainstream research practices or standard interpretation of findings (Estes, 1960;Lykken, 1991;Danziger, 1994;Grice et al, 2006;Smedslund, 2015;Grice et al, 2017). It is possible that the message has been lost partly in the method of presentation, which has for the main part been couched in fairly dry (if thorough and comprehensive) statistical argument and formulae (e.g., Haaf and Rouder, 2019;Williams et al, 2020). At the least, the solutions provided by others to the ergodicity fallacy have clearly lacked the "cut through" to shift mainstream methods of analysis.…”