A Machine Learning Approach to Understanding Patterns of Engagement With Internet-Delivered Mental Health Interventions

Chien, Isabel; Enrique, Ángel; Palacios, Jorge; Regan, Tim; Keegan, Dessie; Carter, David; Tschiatschek, Sebastian; Nori, Aditya V.; Thieme, Anja; Richards, Derek; Doherty, Gavin; Belgrave, Danielle

doi:10.1001/jamanetworkopen.2020.10791

Cited by 104 publications

(109 citation statements)

References 33 publications

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…Study of engagement with digital health apps has been growing, with no consensus yet on ideal construct definitions [ 48 - 50 ]. Simply reporting the number of messages or minutes spent on an app over time may undermine clarity and genuine understanding of the type and manifestation of app utilization related to clinical outcomes of interest [ 51 ]. Further research in this area is warranted.…”

Section: Discussionmentioning

confidence: 99%

A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study

Prochaska¹,

Vogel²,

Chieng³

et al. 2021

J Med Internet Res

147

156

View full text Add to dashboard Cite

Background Misuse of substances is common, can be serious and costly to society, and often goes untreated due to barriers to accessing care. Woebot is a mental health digital solution informed by cognitive behavioral therapy and built upon an artificial intelligence–driven platform to deliver tailored content to users. In a previous 2-week randomized controlled trial, Woebot alleviated depressive symptoms. Objective This study aims to adapt Woebot for the treatment of substance use disorders (W-SUDs) and examine its feasibility, acceptability, and preliminary efficacy. Methods American adults (aged 18-65 years) who screened positive for substance misuse without major health contraindications were recruited from online sources and flyers and enrolled between March 27 and May 6, 2020. In a single-group pre/postdesign, all participants received W-SUDs for 8 weeks. W-SUDs provided mood, craving, and pain tracking and modules (psychoeducational lessons and psychotherapeutic tools) using elements of dialectical behavior therapy and motivational interviewing. Paired samples t tests and McNemar nonparametric tests were used to examine within-subject changes from pre- to posttreatment on measures of substance use, confidence, cravings, mood, and pain. Results The sample (N=101) had a mean age of 36.8 years (SD 10.0), and 75.2% (76/101) of the participants were female, 78.2% (79/101) were non-Hispanic White, and 72.3% (73/101) were employed. Participants’ W-SUDs use averaged 15.7 (SD 14.2) days, 12.1 (SD 8.3) modules, and 600.7 (SD 556.5) sent messages. About 94% (562/598) of all completed psychoeducational lessons were rated positively. From treatment start to end, in-app craving ratings were reduced by half (87/101, 86.1% reporting cravings in the app; odds ratio 0.48, 95% CI 0.32-0.73). Posttreatment assessment completion was 50.5% (51/101), with better retention among those who initially screened higher on substance misuse. From pre- to posttreatment, confidence to resist urges to use substances significantly increased (mean score change +16.9, SD 21.4; P<.001), whereas past month substance use occasions (mean change −9.3, SD 14.1; P<.001) and scores on the Alcohol Use Disorders Identification Test-Concise (mean change −1.3, SD 2.6; P<.001), 10-item Drug Abuse Screening Test (mean change −1.2, SD 2.0; P<.001), Patient Health Questionnaire-8 item (mean change 2.1, SD 5.2; P=.005), Generalized Anxiety Disorder-7 (mean change −2.3, SD 4.7; P=.001), and cravings scale (68.6% vs 47.1% moderate to extreme; P=.01) significantly decreased. Most participants would recommend W-SUDs to a friend (39/51, 76%) and reported receiving the service they desired (41/51, 80%). Fewer felt W-SUDs met most or all of their needs (22/51, 43%). Conclusions W-SUDs was feasible to deliver, engaging, and acceptable and was associated with significant improvements in substance use, confidence, cravings, depression, and anxiety. Study attrition was high. Future research will evaluate W-SUDs in a randomized controlled trial with a more diverse sample and with the use of greater study retention strategies. Trial Registration ClinicalTrials.gov NCT04096001; http://clinicaltrials.gov/ct2/show/NCT04096001.

show abstract

Section: Discussionmentioning

confidence: 99%

A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study

Prochaska¹,

Vogel²,

Chieng³

et al. 2021

J Med Internet Res

147

156

View full text Add to dashboard Cite

show abstract

“…Engagement data may be used within predictive models, providing interpretable and ac tionable outputs (e.g., the need for more frequent therapist contact in order to motivate greater engagement). Chien et al 86 analyzed engagement data from 54,604 patients using a supported online intervention for depression and anxiety. A hidden Markov model was used to identify five engagement subtypes, based on patient interactions with sections of the intervention.…”

Section: Digital Cbtmentioning

confidence: 99%

The promise of machine learning in predicting treatment outcomes in psychiatry

Bondar²,

et al. 2021

Self Cite

View full text Add to dashboard Cite

For many years, psychiatrists have tried to understand factors involved in response to medications or psychotherapies, in order to personalize their treatment choices. There is now a broad and growing interest in the idea that we can develop models to personalize treatment decisions using new statistical approaches from the field of machine learning and applying them to larger volumes of data. In this pursuit, there has been a paradigm shift away from experimental studies to confirm or refute specific hypotheses towards a focus on the overall explanatory power of a predictive model when tested on new, unseen datasets. In this paper, we review key studies using machine learning to predict treatment outcomes in psychiatry, ranging from medications and psychotherapies to digital interventions and neurobiological treatments. Next, we focus on some new sources of data that are being used for the development of predictive models based on machine learning, such as electronic health records, smartphone and social media data, and on the potential utility of data from genetics, electrophysiology, neuroimaging and cognitive testing. Finally, we discuss how far the field has come towards implementing prediction tools in real‐world clinical practice. Relatively few retrospective studies to‐date include appropriate external validation procedures, and there are even fewer prospective studies testing the clinical feasibility and effectiveness of predictive models. Applications of machine learning in psychiatry face some of the same ethical challenges posed by these techniques in other areas of medicine or computer science, which we discuss here. In short, machine learning is a nascent but important approach to improve the effectiveness of mental health care, and several prospective clinical studies suggest that it may be working already.

show abstract

“…This paper presents a novel approach to collecting comprehensive data on treatment progress. The implementation of ML models and AI in behavioral health care is a rapidly moving and innovative field, with the potential to significantly improve screening and clinical outcomes [ 49 , 50 ]. Accurate data can provide more information about the patient and can be translated into clinical decisions faster.…”

Section: Discussionmentioning

confidence: 99%

Leveraging the Power of Nondisruptive Technologies to Optimize Mental Health Treatment: Case Study

Sadeh‐Sharvit¹,

Hollon²

2020

JMIR Ment Health

View full text Add to dashboard Cite

Regular assessment of the effectiveness of behavioral interventions is a potent tool for improving their relevance to patients. However, poor provider and patient adherence characterize most measurement-based care tools. Therefore, a new approach for measuring intervention effects and communicating them to providers in a seamless manner is warranted. This paper provides a brief overview of the available research evidence on novel ways to measure the effects of behavioral treatments, integrating both objective and subjective data. We highlight the importance of analyzing therapeutic conversations through natural language processing. We then suggest a conceptual framework for capitalizing on data captured through directly collected and nondisruptive methodologies to describe the client’s characteristics and needs and inform clinical decision-making. We then apply this context in exploring a new tool to integrate the content of therapeutic conversations and patients’ self-reports. We present a case study of how both subjective and objective measures of treatment effects were implemented in cognitive-behavioral treatment for depression and anxiety and then utilized in treatment planning, delivery, and termination. In this tool, called Eleos, the patient completes standardized measures of depression and anxiety. The content of the treatment sessions was evaluated using nondisruptive, independent measures of conversation content, fidelity to the treatment model, and the back-and-forth of client-therapist dialogue. Innovative applications of advances in digital health are needed to disseminate empirically supported interventions and measure them in a noncumbersome way. Eleos appears to be a feasible, sustainable, and effective way to assess behavioral health care.

show abstract

A Machine Learning Approach to Understanding Patterns of Engagement With Internet-Delivered Mental Health Interventions

Cited by 104 publications

References 33 publications

A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study

A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study

The promise of machine learning in predicting treatment outcomes in psychiatry

Leveraging the Power of Nondisruptive Technologies to Optimize Mental Health Treatment: Case Study

Contact Info

Product

Resources

About