A Calibrated Ensemble Algorithm to Address Data Heterogeneity in Machine Learning: An Application to Identify Severe SLE Flares in Lupus Patients

Abstract

Motivated to address the inconsistency between the essential i.i.d. assumption in machine learning theory and the data heterogeneity in real-world applications, we propose a novel calibrated ensemble (CE) algorithm to facilitate learning with diverse data subgroups. Unlike the traditional ensemble framework in which each learner is trained independently using the entire dataset, our method exploits the strengths of various machine learning models by training them simultaneously and forming modelergonomic data subgroups as part of the training process. Consequently, each learner is calibrated to a unique subset of data based on their individualized predictive strength. Clinically, we can interpret each model as an expert specializing in treating patients with particular disease manifestations. We evaluate the CE model in our motivating domain of identifying lupus patients with severe SLE flares using 1,541 clinical encounters in the Mass General Brigham (MGB) Lupus Cohort. Our experimental results demonstrate the efficacy of our CE model across seven performance evaluation metrics compared to five individual machine learning models and regular ensemble approaches. We further utilize ANOVA and Tukey HSD post-hoc statistical analysis to discover characteristic features of individual model clusters for clinical interpretations.

Machine learning, a new tool for the detection of immunodeficiency patterns in systemic lupus erythematosus

“…37 Recent studies have also appeared for the analysis of SLE using ML techniques, and these works are adjusted to the data available in each of them. [40][41][42]52,53 In a study published by Jorge et al 38 these ML techniques are used to predict hospitalization of SLE patients.…”

Recent advances in the use of machine learning and artificial intelligence to improve diagnosis, predict flares, and enrich clinical trials in lupus

“…Prediction of flares, disease activity, or disease progression with machine learning may help direct therapeutic plans for lupus and suggest specific prophylactic treatment. One study aimed to predict flares from EMRs [19]. Another study similarly estimated the score for different categories of lupus disease activity [20].…”