Identifying psychosis spectrum disorder from experience sampling data using machine learning approaches

Abstract

The ubiquity of smartphones have opened up the possibility of widespread use of the Experience Sampling Method (ESM). The method is used to collect longitudinal data of participants' daily life experiences and is ideal to capture fluctuations in emotions (momentary mental states) as an indicator for later mental ill-health. In this study, ESM data of patients with psychosis spectrum disorder and controls were used to examine daily life emotions and higher order patterns thereof. We attempted to determine whether aggregated ESM data, in which statistical measures represent the distribution and dynamics of the original data, were able to distinguish patients from controls in a predictive modelling framework. Variable importance, recursive feature elimination, and ReliefF methods were used for feature selection. Model training, tuning, and testing were performed in nested cross-validation, based on algorithms such as Random Forests, Support Vector Machines, Gaussian Processes, Logistic Regression and Neural Networks. ROC analysis was used to post-process these models. Stability of model performance was studied using Monte Carlo simulations. The results provide evidence that patterns in emotion changes can be captured by applying a combination of these techniques. Acceleration in the variables anxious and insecure was particularly successful in adding further predictive power to the models. The best results were achieved by Support Vector Machines with radial kernel (accuracy=82% and sensitivity=82%). This proof-of-concept work demonstrates that synergistic machine learning and statistical modeling may be used to harness the power of ESM data in the future.

A metabolite‐based machine learning approach to diagnose Alzheimer‐type dementia in blood: Results from the European Medical Information Framework for Alzheimer disease biomarker discovery cohort

“…The MC simulation conducted with XGBoost, which was the superior predictive model in our analysis, led to a Gaussian distribution of the AUC values according to [11] and as confirmed by Shapiro‐Wilk test ( P value = .6819). The 50 AUC values obtained in MC had a minimum of 0.8614, a maximum of 0.8923, a mean of 0.8761, a median of 0.8766 and a standard deviation of 0.0072.…”

“…This method computes the predictors' importance defined as the standardized Relief score, according to Measuring Predictor Importance chapter of [10]. Part of the prediction modeling methodology in this study was adapted after [11], with different algorithms, and followed recommendations from [10,12]. The analysis was carried out using R software [13].…”

A metabolite-based machine learning approach to diagnose Alzheimer’s-type dementia in blood: Results from the European Medical Information Framework for Alzheimer’s Disease biomarker discovery cohort

“…The MC simulation conducted with XGBoost which was the superior predictive model in our analysis, led to a Gaussian distribution of the AUC values according to [11] and as confirmed by Shapiro-Wilk test (p-value=0.6819). The 50 AUC values obtained in MC had a minimum of 0.8614, a maximum of 0.8923, a mean of 0.8761, a median of 0.8766, and a standard deviation of 0.0072.…”

“…This method computes the predictors’ importance defined as the standardised Relief score, according to Measuring Predictor Importance chapter of [10]. Part of the prediction modelling methodology in this study was adapted after [11], with different algorithms, and followed recommendations from [10, 12]. The analysis was carried out using R software [13].…”

Precision Diagnostic Approach to Predict 5-Year Risk for Microvascular Complications in Type 1 Diabetes

“…For each panel of predictors, the performance was calculated for each round on mean AUROC values from which the optimal number of features was selected. The models were further tested for performance stability using Monte Carlo (MC) simulation consisting of 50 iterations (26,27). Model performance was evaluated with the following metrics: AUROC, prediction accuracy, precision, recall (sensitivity), and F-score.…”