Digital Phenotyping Data to Predict Symptom Improvement and Mental Health App Personalization in College Students: Prospective Validation of a Predictive Model

Abstract

Background Mental health apps offer a transformative means to increase access to scalable evidence-based care for college students. Yet low rates of engagement currently preclude the effectiveness of these apps. One promising solution is to make these apps more responsive and personalized through digital phenotyping methods able to predict symptoms and offer tailored interventions. Objective Following our protocol and using the exact model shared in that paper, our primary aim in this study is to assess the prospective validity of mental health symptom prediction using the mindLAMP app through a replication study. We also explored secondary aims around app intervention personalization and correlations of engagement with the Technology Acceptance Model (TAM) and Digital Working Alliance Inventory scale in the context of automating the study. Methods The study was 28 days in duration and followed the published protocol, with participants collecting digital phenotyping data and being offered optional scheduled and algorithm-recommended app interventions. Study compensation was tied to the completion of weekly surveys and was not otherwise tied to engagement or use of the app. Results The data from 67 participants were used in this analysis. The area under the curve values for the symptom prediction model ranged from 0.58 for the UCLA Loneliness Scale to 0.71 for the Patient Health Questionnaire-9. Engagement with the scheduled app interventions was high, with a study mean of 73%, but few participants engaged with the optional recommended interventions. The perceived utility of the app in the TAM was higher (P=.01) among those completing at least one recommended intervention. Conclusions Our results suggest how digital phenotyping methods can be used to create generalizable models that may help create more personalized and engaging mental health apps. Automating studies is feasible, and our results suggest targets to increase engagement in future studies. International Registered Report Identifier (IRRID) RR2-10.2196/37954

Assessing Digital Phenotyping for App Recommendations and Sustained Engagement: Cohort Study (Preprint)

“…Additionally, our qualitative results suggest facets of a more nuanced picture of engagement beyond metrics like screen time or alliance. Participants agreed that perceived ease of use and perceived utility were important factors in engaging with an app, which aligns with prior research findings grounded in the Technology Acceptance Model [22]. But while these two core factors were necessary for initial engagement, results suggest that sustained engagement requires the addition of habit formation.…”

“…For example, a patient who reports depression while at home may benefit from a cognitive behavioral therapy focused app and another who reports anxiety at work may benefit from a different app offering brief mindfulness exercises. Digital phenotyping methods can also be used to predict changes in anxiety and depression [22], meaning that it may be possible to suggest mental health app use early and as a preventive approach.…”

Increasing the value of digital phenotyping through reducing missingness: a retrospective review and analysis of prior studies

“…For example, participants who feel a greater alliance may be more motivated to complete activities, and we have shown that increased active participation leads to better passive data coverage. However, the relationship between alliance and engagement is complicated 22…”

“…However, the relationship between alliance and engagement is complicated. 22 Combining these methods and focusing on data coverage can lead to positive results. The upward trajectory of data coverage as shown in figure 2, especially the increases in Fall 2021, coincides with the implementation of the strategies discussed above.…”

Digital Phenotyping: Data-Driven Psychiatry to Redefine Mental Health

“…As an example, there is a correlation between circadian rhythm, step counts, or heart rate variability and the diagnosis of a mood disorder or mood episode [45][46][47][48]. Other correlations have been found between data and symptoms of schizophrenia [49][50][51], major depression [52][53][54][55][56][57], mood disorders [46,58,59], posttraumatic stress disorder [60,61], generalized anxiety disorder [62], suicidal thoughts [63,64], sleep disorders [65], addiction [66], stress [53,67], postpartum [68,69], autism [70], and child and adolescent psychiatry [71,72]. Among other examples of the efficiency of DP for prediction or diagnosis in mental health, Instagram photos or Facebook language have been found to be predictors of depression [73,74]; suicidal risk could be assessed from social media [75,76] with increasing precision if DP would integrate electronic health records data [77,78]; automated analysis of free speech can measure relevant mental health changes in emergent psychosis [79] or incoherence in speech in schizophrenia [80].…”