Time‑dependent ROC curve analysis to determine the predictive capacity of seven clinical scales for mortality in patients with COVID‑19: Study of a hospital cohort with very high mortality

Abstract

Clinical data from hospital admissions are typically utilized to determine the prognostic capacity of Coronavirus disease 2019 indices. However, as disease status and severity markers evolve over time, time-dependent receiver operating characteristic (ROC) curve analysis becomes more appropriate. The present analysis assessed predictive power for death at various time points throughout patient hospitalization. In a cohort study involving 515 hospitalized patients (General Hospital Number 1 of Mexican Social Security Institute, Colima, Mexico from February 2021 to December 2022) with COVID-19, seven severity indices [Pneumonia Severity Index (PSI) PaO 2 /FiO 2 arterial oxygen pressure/fraction of inspired oxygen (Kirby index), the Critical Illness Risk Score (COVID-GRAM), the National Early Warning Score 2 (NEWS-2), the quick Sequential Organ Failure Assessment score (qSOFA), the Fibrosis-4 index (FIB-4) and the Viral Pneumonia Mortality Score (MuLBSTA were evaluated using time-dependent ROC curves. Clinical data were collected at admission and at 2, 4, 6 and 8 days into hospitalization. The study calculated the area under the curve (AUC), sensitivity, specificity, and predictive values for each index at these time points. Mortality was 43.9%. Throughout all time points, NEWS-2 demonstrated the highest predictive power for mortality, as indicated by its AUC values. PSI and COVID-GRAM followed, with predictive power increasing as hospitalization duration progressed. Additionally, NEWS-2 exhibited the highest sensitivity (>96% in all periods) but showed low specificity, which increased from 22.9% at admission to 58.1% by day 8. PSI displayed good predictive capacity from admission to day 6 and excellent predictive power at day 8 and its sensitivity remained >80% throughout all periods, with moderate specificity (70.6-77.3%). COVID-GRAM demonstrated good predictive capacity across all periods, with high sensitivity (84.2-87.3%) but low-to-moderate specificity (61.5-67.6%). The qSOFA index initially had poor predictive power upon admission but improved after 4 days. FIB-4 had a statistically significant predictive capacity in all periods (P= 0.001), but with limited clinical value (AUC, 0.639-0.698), and with low sensitivity and specificity. MuLBSTA and IKIRBY exhibited low predictive power at admission and no power after 6 days. In conclusion, in COVID-19 patients with high mortality rates, NEWS-2 and PSI consistently exhibited predictive power for death during hospital stay, with PSI demonstrating the best balance between sensitivity and specificity.