Potential and limitations of machine meta-learning (ensemble) methods for predicting COVID-19 mortality in a large inhospital Brazilian dataset

Abstract

The majority of early prediction scores and methods to predict COVID-19 mortality are bound by methodological flaws and technological limitations (e.g., the use of a single prediction model). Our aim is to provide a thorough comparative study that tackles those methodological issues, considering multiple techniques to build mortality prediction models, including modern machine learning (neural) algorithms and traditional statistical techniques, as well as meta-learning (ensemble) approaches. This study used a dataset from a multicenter cohort of 10,897 adult Brazilian COVID-19 patients, admitted from March/2020 to November/2021, including patients [median age 60 (interquartile range 48–71), 46% women]. We also proposed new original population-based meta-features that have not been devised in the literature. Stacking has shown to achieve the best results reported in the literature for the death prediction task, improving over previous state-of-the-art by more than 46% in Recall for predicting death, with AUROC 0.826 and MacroF1 of 65.4%. The newly proposed meta-features were highly discriminative of death, but fell short in producing large improvements in final prediction performance, demonstrating that we are possibly on the limits of the prediction capabilities that can be achieved with the current set of ML techniques and (meta-)features. Finally, we investigated how the trained models perform on different hospitals, showing that there are indeed large differences in classifier performance between different hospitals, further making the case that errors are produced by factors that cannot be modeled with the current predictors.

Towards Improved XAI-Based Epidemiological Research into the Next Potential Pandemic

“…While the above-mentioned techniques for reducing the number of variables can eliminate redundant and irrelevant features, de Paiva, Pereira, and de Andrade argue that it is not always clear whether these methods result in improvements in the predictive power of ML models [65]. Furthermore, as these methods project the features to a new dimension and the features in the new dimension become mixed features, these new features might not necessarily provide a strong explanatory basis [66].…”

Adaptive Stacking Ensemble Techniques for Early Severity Classification of COVID-19 Patients

“…Ensemble learning, which combines the predictive abilities of two or more base learner models to reduce bias and variance, thereby improving overall prediction performance, has seen recent application in analyzing COVID-19 data. However, determining the optimal combination of models for optimal performance remains challenging, with most researchers manually constructing stacking ensemble models to predict COVID-19's clinical severity [32][33][34][35]. Additionally, some studies opted for simplified severity classifications related only to survival and death, rather than attempting to classify a diverse range of clinical severities [36,37].…”

HOG Ensembled Boosting Machine Learning Approach for Violent Video Classification

“…When this occurs, the boosting algorithm may ignore the vast majority of the training data in favor of the few misclassified observations. As a result, general accuracy in classification tends to drop (17) . In such situation, good solution is using robust boosting (RobustBoost).…”