Compliance with Ecological Momentary Assessment Protocols in Substance Users: a Meta-Analysis

Abstract

Introduction: Ecological Momentary Assessment (EMA) methods allow for real-time data collection in naturalistic environments, and are particularly informative for the examination of substance use which is both time and context dependent. Whilst there are considerable benefits to EMA, poor compliance to assessment protocols has been identified as a limitation, particularly in substance users. Little research has analysed factors which might influence compliance. Methods: The aim of this meta-analysis was to systematically review and meta-analyse potential variables that may influence compliance to EMA protocols in substance users; such as, prompt frequency, total length of assessment period, substance use population and device used to administer EMA prompts. We pre-registered our design, hypotheses and analysis strategy. Results: Following systematic searches of relevant databases we identified k = 128 reported compliance rates in the literature. The pooled compliance rate across all studies was 78.68% (95% CI 76.53%, 80.69%). There was no evidence that any proposed moderators were associated with compliance rates; prompt frequency (Q(3) = 0.98, p = .805) length of assessment period (Q(2) = 1.42, p = .493), substance use population (Q(1) = 1.830, p = .176) or device administration (Q(3) = 4.715, p =.194). Conclusions: The overall compliance rate was similar to that of other fields and recommended rates of compliance (80%), although compliance was not associated with any procedural variables. Furthermore, we identified various limitations in reporting of compliance data and improvement is needed to further elucidate factors which might influence compliance. These findings suggest intensive real-time data collection techniques can be administered in substance using populations, despite previous concerns.

Feasibility of ecological momentary assessment to study depressive symptoms among cancer caregivers

“…Findings from this first study using EMA to understand day‐to‐day markers of depressive symptoms among cancer caregivers suggests that cancer caregivers are amenable to participating in studies with an EMA design and allays potential concerns that these designs may be too burdensome for this population. Although prompt completion rates in the present study did fall closely below the typical rates in prior EMA research, 27 , 28 engagement was not related to factors including caregivers' depressive symptoms, subjective caregiving burden, or the patients' care needs. Findings also support that caregivers' depressive symptoms relate in expected directions with their overall levels of self‐reported day‐to‐day negative and positive affect, although effects were not detected between caregivers' depressive symptoms and their affective variability.…”

“…Our overall engagement of about 60% of prompts completed did not exceed our a priori benchmark of 65%, meaning our recorded engagement was slightly lower than average engagement typical in other EMA studies reported among the general population 27 28 Our engagement was also low compared to a previously published EMA study among cancer caregivers that recorded a 90.5% survey completion rate, though a key difference is that the previously published study examining communication and relationship satisfaction employed a design of twice‐daily prompts over 14 consecutive days 33 .…”

“…To determine recruitment feasibility, an a priori benchmark of 55% enrollment was set based on national norms for recruitment to studies of cancer caregivers 26 . For EMA engagement feasibility, due to a dearth of EMA studies in chronic health populations, an a priori benchmark of 65% prompt completion was set based on national norms for prompt completion in typical EMA studies 27 28 The correlations between survey response completion rate and participants' age, employment status, physical health QOL, caregiving stress, and depressive symptoms and patient ADL/IADL dependency were examined.…”

Does the Combination Matter? Examining the Influence of Alcohol and Cannabis Product Combinations on Simultaneous Use and Consequences in Daily Life

“…Aggregated across the 5 recurring daily surveys (RDS), mean RDS compliance equaled 88.4%, and mean morning survey compliance equaled 81.9%. Both compliance rates exceed the pooled compliance rate shown in a recent meta-analysis on EMA and substance use (Jones et al, 2019). As a part of the parent study, all participants were trained on standardized drink equivalences set forth by the National Institute of Alcoholism and Alcohol Abuse (NIAAA; National Institute on Alcohol Abuse and Alcoholism, 2007).…”

Strengths and Weaknesses of Meta-Analyses

“…Again, critical appraisal involves the systematic evaluation of study quality using a tool such as the Cochrane Collaboration's checklist (Higgins et al, 2011). Furthermore, only a handful (e.g., Jones et al, 2018;Woll & Schönbrodt, in press) have ever been pre-registered 4 , and there is no central collaboration akin to Cochrane, Campbell, or CEE in psychology. Taken together, the evidence suggests that although psychology and related social sciences make frequent use of meta-analyses, these meta-analyses are not often of the high methodological rigor required for the use of the label systematic review.…”