Clinical-learning versus machine-learning for transdiagnostic prediction of psychosis onset in individuals at-risk

Abstract

Predicting the onset of psychosis in individuals at-risk is based on robust prognostic model building methods including a priori clinical knowledge (also termed clinical-learning) to preselect predictors or machine-learning methods to select predictors automatically. To date, there is no empirical research comparing the prognostic accuracy of these two methods for the prediction of psychosis onset. In a first experiment, no improved performance was observed when machine-learning methods (LASSO and RIDGE) were applied—using the same predictors—to an individualised, transdiagnostic, clinically based, risk calculator previously developed on the basis of clinical-learning (predictors: age, gender, age by gender, ethnicity, ICD-10 diagnostic spectrum), and externally validated twice. In a second experiment, two refined versions of the published model which expanded the granularity of the ICD-10 diagnosis were introduced: ICD-10 diagnostic categories and ICD-10 diagnostic subdivisions. Although these refined versions showed an increase in apparent performance, their external performance was similar to the original model. In a third experiment, the three refined models were analysed under machine-learning and clinical-learning with a variable event per variable ratio (EPV). The best performing model under low EPVs was obtained through machine-learning approaches. The development of prognostic models on the basis of a priori clinical knowledge, large samples and adequate events per variable is a robust clinical prediction method to forecast psychosis onset in patients at-risk, and is comparable to machine-learning methods, which are more difficult to interpret and implement. Machine-learning methods should be preferred for high dimensional data when no a priori knowledge is available.

Prediction of the Age at Onset of Spinocerebellar Ataxia Type 3 with Machine Learning

“…Considering the inherent shortage of machine learning, the XGBoost algorithm with "black-box" classifiers seem laborious for clinicians and stakeholders because of its limited interpretability and readability. 25,39 However, the sample size in the present study is the largest SCA3/MJD cohort worldwide, and the XGBoost model still demonstrate valuable performance.…”

“…This approach can be used for model construction by processing a large number of features, capturing nonlinearity and feature interactions, and identifying hard‐to‐recognize patterns in complex data sets automatically. Several machine‐learning algorithms based on AI technology have been widely applied in some medical fields for disease diagnosis, progression assessment, and prognosis evaluation, and these methods have enhanced personalized medical care and optimized treatment strategies 24‐28 …”

“…Several machinelearning algorithms based on AI technology have been widely applied in some medical fields for disease diagnosis, progression assessment, and prognosis evaluation, and these methods have enhanced personalized medical care and optimized treatment strategies. [24][25][26][27][28] In this study, we included the largest sample of affected and presymptomatic SCA3/MJD individuals in mainland China, compared machine-learning approaches versus traditional statistical methods on the prediction of AAO in SCA3/MJD, and developed a disease prediction model for AAO using machine-learning methods. Specifically, we (1) compared the differences in predicting the AAO of SCA3/MJD based on 7 modern machine-learning algorithms, including extreme gradient boosting (XGBoost), adaptive boosting (AdaBoost), random forest (RF), knearest neighbor, support vector regression, elastic network (EN), lasso regression (Lasso), and the traditional LR methods and (2) developed a specific AAO prediction model of SCA3/MJD based on machine-learning algorithms using CAG expansions of 10 polyQ-related genes, sex, and parental origin.…”

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

“…These clinical variables were selected based on a priori knowledge obtained from meta analyses 16,17 , as recommended by the state-of-the-art methodological guidelines 15 . The number of predictors is limited to preserve the Event Per Variable ratio and minimize overfitting biases; including too many variables without a priori filter leads to overfitting problems and poor prognostic accuracy 18 . The method used to develop this model provides similar prognostic accuracy to automatic machine learning methods 18 .…”

“…The number of predictors is limited to preserve the Event Per Variable ratio and minimize overfitting biases; including too many variables without a priori filter leads to overfitting problems and poor prognostic accuracy 18 . The method used to develop this model provides similar prognostic accuracy to automatic machine learning methods 18 .…”