Assessing and Accounting for Measurement in Intensive Longitudinal Studies: Current Practices, Considerations, and Avenues for Improvement

Abstract

Experience sampling methodology studies, in which participants complete self-report questionnaires multiple times a day over an extended period, are increasingly popular for studying dynamics in psychological constructs. To ensure the validity of the results obtained from analyzing the intensive longitudinal data (ILD), greater awareness and understanding of appropriate measurement practices are needed. We surveyed 42 researchers experienced with ILD on how they assess and account for measurement. Results showed that researchers typically use measures not validated specifically for ILD. Assessment of psychometric properties in the participants’ current studies and accounting for these properties when analyzing dynamics was even less common. Moreover, researchers often fail to assess measurement invariance; if tested, the required level of invariance is usually not achieved. However, open science practices appear reasonably well ingrained in ILD studies. We provide recommendations for best practices in ILD studies for creating a solid foundation for measuring and analyzing psychological constructs.

Understanding EMA Data: A Tutorial on Exploring Item Performance in Ecological Momentary Assessment Data

“…A more flexible approach to calculate reliability in longitudinal data with multiple indicators per construct is the use of dynamic factor models (Fuller-Tyszkiewicz et al, 2017), while Schuurman et al (2015) show how to calculate between-and within-person reliability with single-indicator measures. General recommendations regarding reliability in singleand multi-item EMA measures are provided in Vogelsmeier et al (2023).…”

Latent Vector Autoregressive Modeling: A Stepwise Estimation Approach

“…The factor scores represent the positions on the underlying latent variables identified through the factor analysis (DiStefano et al, 2009). Finally, the researchers use the factor scores as observed scores in regular VAR models (Vogelsmeier et al, 2023). This allows researchers to first scrutinize (and potentially adjust) the MM before estimating the parameters of the SM (Bakk et al, 2013;Vermunt, 2010).…”