Rethinking the Dose-Response Relationship Between Usage and Outcome in an Online Intervention for Depression: Randomized Controlled Trial

Abstract

BackgroundThere is now substantial evidence that Web-based interventions can be effective at changing behavior and successfully treating psychological disorders. However, interest in the impact of usage on intervention outcomes has only been developed recently. To date, persistence with or completion of the intervention has been the most commonly reported metric of use, but this does not adequately describe user behavior online. Analysis of alternative measures of usage and their relationship to outcome may help to understand how much of the intervention users may need to obtain a clinically significant benefit from the program.ObjectiveThe objective of this study was to determine which usage metrics, if any, are associated with outcome in an online depression treatment trial.MethodsCardiovascular Risk E-couch Depression Outcome (CREDO) is a randomized controlled trial evaluating an unguided Web-based program (E-couch) based on cognitive behavioral therapy and interpersonal therapy for people with depression and cardiovascular disease. In all, 280 participants in the active arm of the trial commenced the program, delivered in 12 modules containing pages of text and activities. Usage data (eg, number of log-ins, modules completed, time spent online, and activities completed) were captured automatically by the program interface. We estimated the association of these and composite metrics with the outcome of a clinically significant improvement in depression score on the Patient Health Questionnaire (PHQ-9) of ≥5 points.ResultsIn all, 214/280 (76.4%) participants provided outcome data at the end of the 12-week period and were included in the analysis. Of these, 94 (43.9%) participants obtained clinically significant improvement. Participants logged into the program an average of 18.7 times (SD 8.3) with most (62.1%, 133/214) completing all 12 modules. Average time spent online per log-in was 17.3 minutes (SD 10.5). Participants completed an average of 9 of 18 activities available within the program. In a multivariate regression model, only the number of activities completed per log-in was associated with a clinically significant outcome (OR 2.82, 95% CI 1.05-7.59). The final model predicted 7.4% of variance in outcome. Curve estimates indicated that significant logarithmic (P=.009) and linear (P=.002) relationships existed between activities completed per log-in and clinically significant change.ConclusionsOnly one objective measure of usage was independently associated with better outcome of a Web-based intervention of known effectiveness. The 4 usage metrics retained in the final step of the regression accounted for little outcome variance. Medium level users appeared to have little additional benefit compared to low users indicating that assumptions of a linear relationship between use and outcome may be too simplistic and further models and variables need to be explored to adequately understand the relationship.Trial RegistrationAustralian New Zealand Clinical Trials Registry (ANZCTR): ACTRN12610000085077; http:...

The relationship between persuasive technology principles, adherence and effect of web-Based interventions for mental health: A meta-analysis

“…An explanation may be that for both adherence and effectiveness, many factors are at play. Even within one intervention, the relationship between adherence and effectiveness is not always clear cut [19]. In the current study, we have investigated the relationship between adherence and effectiveness in a wide range of interventions.…”

Efficacy of a Web-Based, Crowdsourced Peer-To-Peer Cognitive Reappraisal Platform for Depression: Randomized Controlled Trial

“…In practice, however, many self-guided interventions suffer from high attrition rates and low levels of engagement. A recent review of self-guided, Web-based treatments found a median completion rate of 56% [6]. Open trials show even higher rates of attrition [7].…”

Increased Exposure to Rigid Routines can Lead to Increased Challenging Behavior Following Changes to Those Routines

“…However, findings from several areas (e.g. the impact of environmental risks on children's externalizing behavior) also demonstrate robust associations between factors under investigation and behaviors, in the absence of exposureresponse relationships (Donkin et al, 2013;Fraser, Kirkby, Daniels, Gillroy & Montgomery, 2001;Liu, Leung, McCauley, Ai, Pinto-Martin, 2013;Sinha, Manhar Husain, 2013). Some of the complexity of these relationships is likely to arise because of the multi-level factors that influence behavior.…”