Predicting Depression From Smartphone Behavioral Markers Using Machine Learning Methods, Hyperparameter Optimization, and Feature Importance Analysis: Exploratory Study

Asare, Kennedy Opoku; Terhorst, Yannik; Vega, Julio; Peltonen, Ella; Lagerspetz, Eemil; Ferreira, Denzil

doi:10.2196/26540

Cited by 83 publications

(67 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is not surprising; clinical studies intentionally measure symptoms of a specific serious mental illness (SMI), while non-clinical studies collect measures on more prevalent symptoms across the general population (e.g. depression, stress) [ 3 , 5 , 17 , 20 ]. That being said, symptoms of depression are symptoms of SMIs, including schizophrenia [ 55 ].…”

Section: Discussionmentioning

confidence: 99%

Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies

et al. 2022

View full text Add to dashboard Cite

Mobile sensing data processed using machine learning models can passively and remotely assess mental health symptoms from the context of patients’ lives. Prior work has trained models using data from single longitudinal studies, collected from demographically homogeneous populations, over short time periods, using a single data collection platform or mobile application. The generalizability of model performance across studies has not been assessed. This study presents a first analysis to understand if models trained using combined longitudinal study data to predict mental health symptoms generalize across current publicly available data. We combined data from the CrossCheck (individuals living with schizophrenia) and StudentLife (university students) studies. In addition to assessing generalizability, we explored if personalizing models to align mobile sensing data, and oversampling less-represented severe symptoms, improved model performance. Leave-one-subject-out cross-validation (LOSO-CV) results were reported. Two symptoms (sleep quality and stress) had similar question-response structures across studies and were used as outcomes to explore cross-dataset prediction. Models trained with combined data were more likely to be predictive (significant improvement over predicting training data mean) than models trained with single-study data. Expected model performance improved if the distance between training and validation feature distributions decreased using combined versus single-study data. Personalization aligned each LOSO-CV participant with training data, but only improved predicting CrossCheck stress. Oversampling significantly improved severe symptom classification sensitivity and positive predictive value, but decreased model specificity. Taken together, these results show that machine learning models trained on combined longitudinal study data may generalize across heterogeneous datasets. We encourage researchers to disseminate collected de-identified mobile sensing and mental health symptom data, and further standardize data types collected across studies to enable better assessment of model generalizability.

show abstract

Section: Discussionmentioning

confidence: 99%

Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Equipping smartphones and wearables to capture moment-by-moment datasets with sensing apps has made it possible to collect datasets passively and in naturalistic settings. Inherent in these datasets are behavioral patterns: routines, rhythms, activities, and interactions that are useful indicators of depression [ 61 ].…”

Section: Data Miningmentioning

confidence: 99%

Machine Learning-Based Behavioral Diagnostic Tools for Depression: Advances, Challenges, and Future Directions

Richter

Fishbain

Richter‐Levin

et al. 2021

JPM

View full text Add to dashboard Cite

The psychiatric diagnostic procedure is currently based on self-reports that are subject to personal biases. Therefore, the diagnostic process would benefit greatly from data-driven tools that can enhance accuracy and specificity. In recent years, many studies have achieved promising results in detecting and diagnosing depression based on machine learning (ML) analysis. Despite these favorable results in depression diagnosis, which are primarily based on ML analysis of neuroimaging data, most patients do not have access to neuroimaging tools. Hence, objective assessment tools are needed that can be easily integrated into the routine psychiatric diagnostic process. One solution is to use behavioral data, which can be easily collected while still maintaining objectivity. The current paper summarizes the main ML-based approaches that use behavioral data in diagnosing depression and other psychiatric disorders. We classified these studies into two main categories: (a) laboratory-based assessments and (b) data mining, the latter of which we further divided into two sub-groups: (i) social media usage and movement sensors data and (ii) demographic and clinical information. The paper discusses the advantages and challenges in this field and suggests future research directions and implementations. The paper’s overarching aim is to serve as a first step in synthetizing existing knowledge about ML-based behavioral diagnosis studies in order to develop interventions and individually tailored treatments in the future.

show abstract

“…In another study, Saeb et al [ 32 ] found significant correlations between depression and passive data such as phone use and GPS in a sample of 40 participants. Asare et al [ 33 ] found that age group and gender as predictors led to improved machine learning performance. Their study concluded that behavioral markers indicative of depression can be unobtrusively identified using smartphone sensor data [ 33 ].…”

Section: Introductionmentioning

confidence: 99%

A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study

Choudhary¹,

Thomas²,

Ellenberger³

et al. 2022

JMIR Form Res

View full text Add to dashboard Cite

Background Depression is a major global cause of morbidity, an economic burden, and the greatest health challenge leading to chronic disability. Mobile monitoring of mental conditions has long been a sought-after metric to overcome the problems associated with the screening, diagnosis, and monitoring of depression and its heterogeneous presentation. The widespread availability of smartphones has made it possible to use their data to generate digital behavioral models that can be used for both clinical and remote screening and monitoring purposes. This study is novel as it adds to the field by conducting a trial using private and nonintrusive sensors that can help detect and monitor depression in a continuous, passive manner. Objective This study demonstrates a novel mental behavioral profiling metric (the Mental Health Similarity Score), derived from analyzing passively monitored, private, and nonintrusive smartphone use data, to identify and track depressive behavior and its progression. Methods Smartphone data sets and self-reported Patient Health Questionnaire-9 (PHQ-9) depression assessments were collected from 558 smartphone users on the Android operating system in an observational study over an average of 10.7 (SD 23.7) days. We quantified 37 digital behavioral markers from the passive smartphone data set and explored the relationship between the digital behavioral markers and depression using correlation coefficients and random forest models. We leveraged 4 supervised machine learning classification algorithms to predict depression and its severity using PHQ-9 scores as the ground truth. We also quantified an additional 3 digital markers from gyroscope sensors and explored their feasibility in improving the model’s accuracy in detecting depression. Results The PHQ-9 2-class model (none vs severe) achieved the following metrics: precision of 85% to 89%, recall of 85% to 89%, F1 of 87%, and accuracy of 87%. The PHQ-9 3-class model (none vs mild vs severe) achieved the following metrics: precision of 74% to 86%, recall of 76% to 83%, F1 of 75% to 84%, and accuracy of 78%. A significant positive Pearson correlation was found between PHQ-9 questions 2, 6, and 9 within the severely depressed users and the mental behavioral profiling metric (r=0.73). The PHQ-9 question-specific model achieved the following metrics: precision of 76% to 80%, recall of 75% to 81%, F1 of 78% to 89%, and accuracy of 78%. When a gyroscope sensor was added as a feature, the Pearson correlation among questions 2, 6, and 9 decreased from 0.73 to 0.46. The PHQ-9 2-class model+gyro features achieved the following metrics: precision of 74% to 78%, recall of 67% to 83%, F1 of 72% to 78%, and accuracy of 76%. Conclusions Our results demonstrate that the Mental Health Similarity Score can be used to identify and track depressive behavior and its progression with high accuracy.

show abstract

Predicting Depression From Smartphone Behavioral Markers Using Machine Learning Methods, Hyperparameter Optimization, and Feature Importance Analysis: Exploratory Study

Cited by 83 publications

References 87 publications

Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies

Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies

Machine Learning-Based Behavioral Diagnostic Tools for Depression: Advances, Challenges, and Future Directions

A Machine Learning Approach for Detecting Digital Behavioral Patterns of Depression Using Nonintrusive Smartphone Data (Complementary Path to Patient Health Questionnaire-9 Assessment): Prospective Observational Study

Contact Info

Product

Resources

About