Compliance With Mobile Ecological Momentary Assessment Protocols in Children and Adolescents: A Systematic Review and Meta-Analysis

Abstract

BackgroundMobile device-based ecological momentary assessment (mobile-EMA) is increasingly used to collect participants' data in real-time and in context. Although EMA offers methodological advantages, these advantages can be diminished by participant noncompliance. However, evidence on how well participants comply with mobile-EMA protocols and how study design factors associated with participant compliance is limited, especially in the youth literature.ObjectiveTo systematically and meta-analytically examine youth’s compliance to mobile-EMA protocols and moderators of participant compliance in clinical and nonclinical settings.MethodsStudies using mobile devices to collect EMA data among youth (age ≤18 years old) were identified. A systematic review was conducted to describe the characteristics of mobile-EMA protocols and author-reported factors associated with compliance. Random effects meta-analyses were conducted to estimate the overall compliance across studies and to explore factors associated with differences in youths’ compliance.ResultsThis review included 42 unique studies that assessed behaviors, subjective experiences, and contextual information. Mobile phones were used as the primary mode of EMA data collection in 48% (20/42) of the reviewed studies. In total, 12% (5/42) of the studies used wearable devices in addition to the EMA data collection platforms. About half of the studies (62%, 24/42) recruited youth from nonclinical settings. Most (98%, 41/42) studies used a time-based sampling protocol. Among these studies, most (95%, 39/41) prompted youth 2-9 times daily, for a study length ranging from 2-42 days. Sampling frequency and study length did not differ between studies with participants from clinical versus nonclinical settings. Most (88%, 36/41) studies with a time-based sampling protocol defined compliance as the proportion of prompts to which participants responded. In these studies, the weighted average compliance rate was 78.3%. The average compliance rates were not different between studies with clinical (76.9%) and nonclinical (79.2%; P=.29) and studies that used only a mobile-EMA platform (77.4%) and mobile platform plus additional wearable devices (73.0%, P=.36). Among clinical studies, the mean compliance rate was significantly lower in studies that prompted participants 2-3 times (73.5%) or 4-5 times (66.9%) compared with studies with a higher sampling frequency (6+ times: 89.3%). Among nonclinical studies, a higher average compliance rate was observed in studies that prompted participants 2-3 times daily (91.7%) compared with those that prompted participants more frequently (4-5 times: 77.4%; 6+ times: 75.0%). The reported compliance rates did not differ by duration of EMA period among studies from either clinical or nonclinical settings.ConclusionsThe compliance rate among mobile-EMA studies in youth is moderate but suboptimal. Study design may affect protocol compliance differently between clinical and nonclinical participants; including additional wearable devices did not affect...

Compliance with ecological momentary assessment protocols in substance users: a meta‐analysis

“…Our meta-regressions identified that the presence of event related assessments increased compliance, and that participants with a diagnosis of substance use disorder had lower compliance rates. The pooled compliance estimate was similar to those obtained from similar syntheses of EMA studies in other fields (children and adolescents [41]), and higher than those in others (physical activity and dietary behaviours [40]. The primary aim of this meta-analysis was to provide a pooled estimate of compliance for EMA studies into substance use to provide a reference point for future research.…”

“…Random-effects models were used due to considerable heterogeneity, and we used restricted maximum likelihood as they are more conservative than maximum likelihood models [51] and perform better than other random-effects models [52]. As compliance rates are clustered within participants we adjusted the sample size by the within-and between-study variance (intraclass correlation coefficient), similar to the method described by Wen et al [41]). Square-root arcsine transformations were used rather than raw proportions to reduce variance instability and preclude confidence intervals falling outside the range of 0.0-1.0 [53].…”

“…In a narrative review, Shiffman et al [8] reported compliance rates ranging from 50 to 90%, but stated: 'it is unclear what caused this substantial variation'. A recent meta-analysis examining compliance in EMA studies assessing behaviours, subjective experiences and contextual information in children and adolescents reported an overall compliance rate of 78.3% [41], with no difference between clinical and non-clinical samples. A recent meta-analysis examining compliance in EMA studies assessing behaviours, subjective experiences and contextual information in children and adolescents reported an overall compliance rate of 78.3% [41], with no difference between clinical and non-clinical samples.…”

Drug use patterns and predictors among homeless youth: Results of an ecological momentary assessment

“…EMA methods may further our understanding of contextual factors related to realtime behaviors and combat some of the limitations related to recall bias by using repeated assessments that allow the participant to provide data proximal to the time of the behavior and that can account for current contextual factors such as housing and urge (14,22). EMA is the most accurate way to measure real-time factors in natural settings, is considered the gold standard (23,24), has reached high youth compliance rates (78%) across 42 studies (25), and has been used to assess drinking behaviors (26) and drug use among youth. To date, no studies have used EMA to assess drug use in real-time among homeless youth or to determine the real-time predictors of drug use in this high risk population.…”

Combining Continuous Smartphone Native Sensors Data Capture and Unsupervised Data Mining Techniques for Behavioral Changes Detection: A Case Series of the Evidence-Based Behavior (eB2) Study (Preprint)

“…E-health involves the integration of new technologies into routine clinical practice by increasing networking possibilities between patient and clinicians. Recent trials [1] using mobile electronic devices have proven successful real-world monitoring in real time, and improved assessment possibilities in a large panel of clinical settings. The assessment of the dynamic relationships among patients between events and disease course is enhanced by the development of momentary data collection strategies, such as experience sampling methods (ESM) and ecological momentary assessment (EMA).…”