Are ecological momentary assessment measures of intervention change worth the trouble? Examining rumination as a case-in-point

Abstract

Objective: Ecological momentary assessment (EMA) is increasingly being incorporated into intervention studies to acquire a more fine-grained and ecologically valid assessment of change, while minimizing recall bias relative to conventional retrospective self-report questionnaires. The added utility of including relatively burdensome EMA measures of change in a clinical trial hinges on several psychometric assumptions, including that these measure are: (1) reliable, (2) related but not redundant with conventional self-report measures (convergent and discriminant validity), (3) sensitive to intervention-related change, and (4) associated with a clinically-relevant criterion of patient improvement (criterion validity) above and beyond conventional self-report measures (incremental validity). Methods: Using data from 4 trials (N = 412) of app-based meditation training we examined the reliability, validity, and sensitivity to change of conventional self-report and EMA measures of improvement in rumination.Results: Conventional self-report and EMA measures of rumination were only modestly correlated, particularly with regards to change over time, which may be due at least in part to the substantially lower reliability of change scores. Larger changes in rumination were observed for the conventional self-report measures relative to EMA. Notably, change in both self-report and EMA rumination accounted for unique variance in overall depressive symptom improvement, demonstrating incremental predictive validity.Conclusions: Conventional self-report and EMA measures of rumination provide distinct and clinically meaningful information. Researchers using EMA should carefully consider the psychometric properties of their measures and the precise construct they intend to capture.