Background Depression is an extremely prevalent issue in the United States, with an estimated 7% of adults experiencing at least one major depressive episode in 2017. Although psychotherapy and medication management are effective treatments for depression, significant barriers in accessing care persist. Virtual care can potentially address some of these obstacles. Objective We conducted a preliminary investigation of utilization characteristics and effectiveness of an on-demand health system for reducing depression symptoms. Methods Data were analyzed from 1662 users of an on-demand mental health system that includes behavioral health coaching, clinical services (therapy and psychiatry), and self-guided content and assessments primarily via a mobile app platform. Measures included engagement characterized by mobile app data, member satisfaction scores collected via in-app surveys, and depression symptoms via the Patient Health Questionnaire-2 (PHQ-2) at baseline and 8-12 week follow-up. Descriptive statistics are reported for measures, and pre/post-PHQ-2 data were analyzed using the McNemar test. A chi-square test was used to test the association between the proportion of individuals with an improvement in PHQ-2 result and care modality (coaching, therapy, and psychiatry, or hybrid). Results During the study period, 65.5% of individuals (1089/1662) engaged only in coaching services, 27.6% of individuals (459/1662) were engaged in both coaching and clinical services, 3.3% of individuals (54/1662) engaged only in clinical services, and 3.7% of individuals (61/1662) only used the app. Of the 1662 individuals who completed the PHQ-2 survey, 772 (46.5%) were considered a positive screen at intake, and 890 (53.6%) were considered a negative screen at intake. At follow-up, 477 (28.7%) of individuals screened positive, and 1185 (71.3%) screened negative. A McNemar test showed that there was a statistically significant decrease in the proportion of users experiencing depressed mood and anhedonia more than half the time at follow-up (P<.001). A chi-square test showed there was no significant association between care modality and the proportion of individuals with an improvement in PHQ-2 score. Conclusions This study provides preliminary insights into which aspects of an on-demand mental health system members are utilizing and levels of engagement and satisfaction over an 8-12 week window. Additionally, there is some signal that this system may be useful for reducing depression symptoms in users over this period. Additional research is required, given the study limitations, and future research directions are discussed.

Background Anxiety is an extremely prevalent condition, and yet, it has received notably less attention than depression and other mental health conditions from a research, clinical, and public health perspective. The COVID-19 pandemic has only exacerbated growing concerns about the burden of anxiety due to the confluence of physical health risks, economic stressors, social isolation, and general disruption of daily activities. Objective This study examines differences in anxiety outcomes by care modality (coaching, teletherapy and telepsychiatry, and combined care) within an on-demand mental health system. We also explore the association between levels of engagement within each care modality and odds of improvement in symptoms of anxiety. Methods We conducted a retrospective observational study of individuals who accessed Ginger, an on-demand mental health system. Data were collected from 1611 Ginger members between January 1, 2018, and December 31, 2019. We used logistic regression to assess the association between care modality and improvement in anxiety symptoms. Within each modality, we assessed the association between level of engagement and improvement. Results Of 1611 Ginger members, 761 (47.0%) experienced a decrease in anxiety symptoms, as measured by a change from a positive to a negative 2-item Generalized Anxiety Disorder (GAD-2) screen. Among members who still screened positive at follow-up (865/1611, 53%), a total of 192 members (11.9%) experienced a clinically significant score reduction in the full GAD-7 (ie, a score reduction of >5 points), even though their GAD-2 scores were still positive. All modalities showed increased odds of improvement compared to those who were not engaged with coaching or clinical services (“app-only”). Higher GAD-7 intake scores were also associated with decreased odds of improvement. Conclusions This study found increased odds of anxiety improvement for all care modalities compared to those who did not engage in care, with larger effect sizes for higher utilization within all care modalities. Additionally, there is a promising observation that those engaged in combined care (teletherapy and text-based coaching) had the greatest odds of anxiety improvement. Future directions include more detailed classifications of utilization patterns and an exploration of explanations and solutions for lower-utilization members.

BACKGROUND Recommender systems have great potential in mental health care to provide self-guided content to supplement the mental health journey for patients scalably; however, traditional filtering approaches often have skewed input data distributions, are static or may not account for changes in symptoms and clinical presentation. In this study, we describe and evaluate two knowledge-based content recommendation system models as part of Ginger, an on-demand mental health platform that seek to address the issues above. OBJECTIVE In this study, we describe and evaluate two knowledge-based content recommendation system models as part of Ginger, an on-demand mental health platform that seek to address the issues above. METHODS We developed two models to provide content recommendations for the Ginger mental health app content. First, a method that uses members' responses to onboarding questions to recommend content cards from the content library of the mental health platform. Second, a dynamic conversation-based recommendation method that matches the semantic similarity between the content of a conversation and the text description of content cards to make recommendations suitable for a conversational snippet. As a measure of success for these recommendation models, we examined the relevance of content cards to members’ conversations with their coach and completion rates of selected content within the app measured over 14018 users. RESULTS Conversation-based recommendation models performed better than random recommendations for all lengths of conversational sessions considering fractions of relevant cards as well as very relevant cards. In an offline analysis, conversation-based recommendations had a 16.1% higher relevance rate (for the top 5 recommended cards) averaged across sessions of varying lengths as compared to a random control algorithm. Comparing completion rates of content delivered in the app to over 14018 users, conversation-based content recommendations that had 11.4% higher completion rates per card than onboarding response based recommendations (P=.003) and 26.1% higher than random recommendations (P=.005)). CONCLUSIONS Recommender systems can help scale and supplement digital mental health care with relevant content and self-care recommendations. Conversation-based recommendation models allow for dynamic recommendations based on information gathered during the course of text-based coaching sessions, which is a critical capability given the changing nature of mental health over the course of treatment.

UNSTRUCTURED Mental health is a growing public health priority in the United States and globally. Measurement-based care (MBC) has been shown to improve outcomes in clinical care, yet there are significant challenges to implementation. New technologies offer opportunities to address these obstacles and to measure new and existing constructs at a scale that was previously not possible. This paper aims to summarize existing literature on MBC, focusing on mental health and digital health. Specifically, we describe a case example called Ginger, a novel on-demand virtual system for delivering mental health services, and demonstrate how this platform aligns with core principles of MBC in mental health based on existing frameworks and findings from the literature. Additionally, we integrate feedback from multidisciplinary stakeholders (clinical practitioners, data science, product development and management, and research) to develop a perspective on key tenets of measurement in this system. To navigate the challenges of translating traditional measurement tools into new technologies in addition to developing new measurements, this multidisciplinary perspective is required. Ultimately, this will enhance our understanding of mental health and ability to develop interventions to improve outcomes.

BACKGROUND Depression is an extremely prevalent issue in the United States, with an estimated 7% of adults experiencing at least one major depressive episode in 2017. Although psychotherapy and medication management are effective treatments for depression, significant barriers in accessing care persist. Virtual care can potentially address some of these obstacles. OBJECTIVE We conducted a preliminary investigation of utilization characteristics and effectiveness of an on-demand health system for reducing depression symptoms. METHODS Data were analyzed from 1662 users of an on-demand mental health system that includes behavioral health coaching, clinical services (therapy and psychiatry), and self-guided content and assessments primarily via a mobile app platform. Measures included engagement characterized by mobile app data, member satisfaction scores collected via in-app surveys, and depression symptoms via the Patient Health Questionnaire-2 (PHQ-2) at baseline and 8-12 week follow-up. Descriptive statistics are reported for measures, and pre/post-PHQ-2 data were analyzed using the McNemar test. A chi-square test was used to test the association between the proportion of individuals with an improvement in PHQ-2 result and care modality (coaching, therapy, and psychiatry, or hybrid). RESULTS During the study period, 65.5% of individuals (1089/1662) engaged only in coaching services, 27.6% of individuals (459/1662) were engaged in both coaching and clinical services, 3.3% of individuals (54/1662) engaged only in clinical services, and 3.7% of individuals (61/1662) only used the app. Of the 1662 individuals who completed the PHQ-2 survey, 772 (46.5%) were considered a positive screen at intake, and 890 (53.6%) were considered a negative screen at intake. At follow-up, 477 (28.7%) of individuals screened positive, and 1185 (71.3%) screened negative. A McNemar test showed that there was a statistically significant decrease in the proportion of users experiencing depressed mood and anhedonia more than half the time at follow-up (<i>P</i>&lt;.001). A chi-square test showed there was no significant association between care modality and the proportion of individuals with an improvement in PHQ-2 score. CONCLUSIONS This study provides preliminary insights into which aspects of an on-demand mental health system members are utilizing and levels of engagement and satisfaction over an 8-12 week window. Additionally, there is some signal that this system may be useful for reducing depression symptoms in users over this period. Additional research is required, given the study limitations, and future research directions are discussed.