A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score (Preprint)

Abstract

BACKGROUND Several models have been developed to predict mortality in patients with Covid-19 pneumonia, but only few have demonstrated enough discriminatory capacity. Machine-learning algorithms represent a novel approach for data-driven prediction of clinical outcomes with advantages over statistical modelling. OBJECTIVE To developed the Piacenza score, a Machine-learning based score, to predict 30-day mortality in patients with Covid-19 pneumonia METHODS The study comprised 852 patients with COVID-19 pneumonia, admitted to the Guglielmo da Saliceto Hospital (Italy) from February to November 2020. The patients’ medical history, demographic and clinical data were collected in an electronic health records. The overall patient dataset was randomly splitted into derivation and test cohort. The score was obtained through the Naïve Bayes classifier and externally validated on 86 patients admitted to Centro Cardiologico Monzino (Italy) in February 2020. Using a forward-search algorithm six features were identified: age; mean corpuscular haemoglobin concentration; PaO2/FiO2 ratio; temperature; previous stroke; gender. The Brier index was used to evaluate the ability of ML to stratify and predict observed outcomes. A user-friendly web site available at (https://covid.7hc.tech.) was designed and developed to enable a fast and easy use of the tool by the final user (i.e., the physician). Regarding the customization properties to the Piacenza score, we added a personalized version of the algorithm inside the website, which enables an optimized computation of the mortality risk score for a single patient, when some variables used by the Piacenza score are not available. In this case, the Naïve Bayes classifier is re-trained over the same derivation cohort but using a different set of patient’s characteristics. We also compared the Piacenza score with the 4C score and with a Naïve Bayes algorithm with 14 features chosen a-priori. RESULTS The Piacenza score showed an AUC of 0.78(95% CI 0.74-0.84 Brier-score 0.19) in the internal validation cohort and 0.79(95% CI 0.68-0.89, Brier-score 0.16) in the external validation cohort showing a comparable accuracy respect to the 4C score and to the Naïve Bayes model with a-priori chosen features, which achieved an AUC of 0.78(95% CI 0.73-0.83, Brier-score 0.26) and 0.80(95% CI 0.75-0.86, Brier-score 0.17) respectively. CONCLUSIONS A personalized Machine-learning based score with a purely data driven features selection is feasible and effective to predict mortality in patients with COVID-19 pneumonia.

COVID-19 Time of Intubation Mortality Evaluation (C-TIME): A System for Predicting Mortality of Patients with COVID-19 Pneumonia at the Time They Require Mechanical Ventilation

“…The discriminant accuracy achieved by C-TIME was modest, although similar to several other COVID-19 mortality prediction systems with AUROCs ranging 0.72-0.79 (5,10,17,19,21,38). We believe that it is inherently difficult to predict COVID-19 mortality at the time of intubation because such patients are relatively clinical homogeneous; most have life-threatening, single organ, respiratory failure (see table 1…”

“…Our sensitivity analysis showed that the later did not affect our AUROC estimates. Our EMR data source limited our ability to include variables not recorded as discrete data, such as COVID-19 vaccination status and pre-existing atrial fibrillation.The discriminant accuracy achieved by C-TIME was modest, although similar to several other COVID-19 mortality prediction systems with AUROCs ranging 0.72-0.79(5,10,17,19,21,38). We believe that it is inherently difficult to predict COVID-19 mortality at the time of intubation because such patients are relatively clinical homogeneous; most have life-threatening, single organ, respiratory failure (see table 1) (3).…”

Unraveling complex relationships between COVID-19 risk factors using machine learning based models for predicting mortality of hospitalized patients and identification of high-risk group: a large retrospective study

“…However, future studies are required to investigate the impact and interaction of different risk factors that were not included in the dataset of this study. Other studies also have reported the better performance of ML algorithms for investigating COVID-19 stratification, mortality risk, and identification of high-risk patients (6,50).…”

“…The identification of contributing factors would allow for applying targeted strategies in patients with the highest mortality risk. Patients' accurate history, clinical signs, fever and oxygen saturation measurements, blood cell counts (CBCs), other laboratory findings, computed tomography (CT) scan imaging, and real-time reversetranscription polymerase chain reaction (RT-PCR) test are included in prognostic and diagnostic criteria (5,6). According to the previous studies, some laboratory findings such as lymphopenia, neutropenia, increased alanine aminotransferase (ALT), aspartate aminotransferase (AST), lactate dehydrogenase (LDH), highsensitivity C-reactive protein (hs-CRP), and some clinical signs such as myalgia and shortness of breath had a relationship with an increased mortality and could also be considered as risk factors for COVID-19 mortality (7,8).…”