Selection of the Number of Participants in Intensive Longitudinal Studies: A User-friendly Shiny App and Tutorial to Perform Power Analysis in Multilevel Regression Models that Account for Temporal Dependencies

Abstract

In recent years the popularity of procedures to collect intensive longitudinal data, such as the Experience Sampling Method, has immensely increased. The data collected using such designs allow researchers to study the dynamics of psychological functioning, and how these dynamics differ across individuals. To this end, the data are often modeled with multilevel regression models. An important question that arises when designing intensive longitudinal studies is how to determine the number of participants needed to test specific hypotheses regarding the parameters of these models with sufficient power. Power calculations for intensive longitudinal studies are challenging, because of the hierarchical data structure in which repeated observations are nested within the individuals and because of the serial dependence that is typically present in this data. We, therefore, present a user-friendly application and step-by-step tutorial to perform simulation-based power analyses for a set of models that are popular in intensive longitudinal research. Since many studies use the same sampling protocol (i.e., a fixed number of at least approximately equidistant observations) within individuals, we assume this protocol fixed and focus on the number of participants. All included models explicitly account for the temporal dependencies in the data by assuming serially correlated errors or including autoregressive effects.

Predicting Sexual Desire in Daily Life from an Attachment Perspective: An Experience Sampling study

“…In the present study, the hypothesized moderating effect of emotional intimacy on the association between stress and sexual desire (Dewitte, 2012;Mikulincer & Shaver, 2007) was not found despite high statistical power (90%) (Lafit et al, 2020). A possible explanation could be that the emotion-regulation mechanism, by which stress activates intimacy needs, may be activated more frequently in distressed or dysfunctional couples than in the present healthy community sample (Brandão et al, 2019;Dub e, Corsini-Munt, Muise, & Rosen, 2019;Rick, Falconier, & Wittenborn, 2017).…”

“…Statistical power for a cross-level interaction effect was calculated using the R Shiny web application (Lafit et al, 2020). Based on the analyses (effect size of d ¼ 0.06, with 134 participants with (on average) 47 observations per participant) the power for detecting this 2-way interaction was 90%.…”

The dynamics, processes, mechanisms, and functioning of personality: An overview of the field

“…For example, in multilevel models that are typically fitted in ESM studies, statistical power depends on many parameters such as the number of persons and assessments, random slope variance, intraclass correlation, and fixed effects. For relevant guides and tools, see, for example, Arend and Sch€ afer (2019), Scherbaum andPesner (2019), andLafit et al (2020). Generally, power analyses should be tailored to the type of effect of interest.…”

Incorporating Ecological Momentary Assessment into multi-method investigations of cognitive aging: Promise and practical considerations

“…As such, researchers will need power calculations to inform decisions pertaining to study duration and sample size. Although conducting power analyses for complex, hierarchically-nested data, like that collected via EMA, can be difficult, recent work has begun to provide open-access materials specifically tailored towards investigators using EMA that offer guidance and analytical pipelines for conducting power analyses (Kirtley et al, in press;Lafit et al, 2020). From a practical standpoint, a priori power analyses are in the researcher's best interest in order to prevent the unnecessary oversampling of (and burden to) participants as well as the waste of resources.…”