Measurement invariance within and between individuals: a distinct problem in testing the equivalence of intra- and inter-individual model structures

Abstract

We address the question of equivalence between modeling results obtained on intra-individual and inter-individual levels of psychometric analysis. Our focus is on the concept of measurement invariance and the role it may play in this context. We discuss this in general against the background of the latent variable paradigm, complemented by an operational demonstration in terms of a linear state-space model, i.e., a time series model with latent variables. Implemented in a multiple-occasion and multiple-subject setting, the model simultaneously accounts for intra-individual and inter-individual differences. We consider the conditions—in terms of invariance constraints—under which modeling results are generalizable (a) over time within subjects, (b) over subjects within occasions, and (c) over time and subjects simultaneously thus implying an equivalence-relationship between both dimensions. Since we distinguish the measurement model from the structural model governing relations between the latent variables of interest, we decompose the invariance constraints into those that involve structural parameters and those that involve measurement parameters and relate to measurement invariance. Within the resulting taxonomy of models, we show that, under the condition of measurement invariance over time and subjects, there exists a form of structural equivalence between levels of analysis that is distinct from full structural equivalence, i.e., ergodicity. We demonstrate how measurement invariance between and within subjects can be tested in the context of high-frequency repeated measures in personality research. Finally, we relate problems of measurement variance to problems of non-ergodicity as currently discussed and approached in the literature.

Integrating structure and dynamics in personality assessment: First steps toward the development and validation of a personality dynamics diary.

“…Our paper shows that this may be possible, although many questions remain open. For example, from a psychometric perspective, it will be important to clarify the extent to which within-person factor structures differ between persons (Adolf, Schuurman, Borkenau, Borsboom, & Dolan, 2014;Molenaar & Campbell, 2009), and whether such differences can result in biased estimates of within-person correlations. It also will be important to clarify the number of data points that are necessary to obtain reliable estimates of within-person correlations (Mejía, Hooker, Ram, Pham, & Metoyer, 2014).…”

A Review of Current Ambulatory Assessment Studies in Adolescent Samples and Practical Recommendations

“…Yet, future methodological and theoretical research is needed to establish the best practices for studying youth in “the wild.” First, we need to further examine how psychological processes can be best studied in daily life. With regard to construct validation, additional studies are needed to develop, test, and establish instruments that are brief, yet reliable and valid at the between‐person and within‐person level (e.g., see Adolf, Schuurman, Borkenau, Borsboom, & Dolan, ; Brose, Schmiedek, Koval, & Kuppens, ; Schuurman & Hamaker, ). Relatedly, more work needs to be done on how best to assess reliability and validity for measures that are used in intensive longitudinal designs.…”

A Comparison of Inverse-Wishart Prior Specifications for Covariance Matrices in Multilevel Autoregressive Models

“…It is likely that many of these dynamical processes also differ across individuals (see for instance Adolf, Schuurman, Borkenau, Borsboom, & Dolan, 2014;Hamaker, 2012;Lodewyckx, Tuerlinckx, Kuppens, Allen, & Sheeber, 2011;Molenaar, 2004;Rovine & Walls, 2006;Wang, Hamaker, & Bergeman, 2012). For instance, stressful situations may strongly affect the mood of one individual, while they have little effect on the mood of another individual.…”