Prognostic models in COVID-19 infection that predict severity: a systematic review

Abstract

Current evidence on COVID-19 prognostic models is inconsistent and clinical applicability remains controversial. We performed a systematic review to summarize and critically appraise the available studies that have developed, assessed and/or validated prognostic models of COVID-19 predicting health outcomes. We searched six bibliographic databases to identify published articles that investigated univariable and multivariable prognostic models predicting adverse outcomes in adult COVID-19 patients, including intensive care unit (ICU) admission, intubation, high-flow nasal therapy (HFNT), extracorporeal membrane oxygenation (ECMO) and mortality. We identified and assessed 314 eligible articles from more than 40 countries, with 152 of these studies presenting mortality, 66 progression to severe or critical illness, 35 mortality and ICU admission combined, 17 ICU admission only, while the remaining 44 studies reported prediction models for mechanical ventilation (MV) or a combination of multiple outcomes. The sample size of included studies varied from 11 to 7,704,171 participants, with a mean age ranging from 18 to 93 years. There were 353 prognostic models investigated, with area under the curve (AUC) ranging from 0.44 to 0.99. A great proportion of studies (61.5%, 193 out of 314) performed internal or external validation or replication. In 312 (99.4%) studies, prognostic models were reported to be at high risk of bias due to uncertainties and challenges surrounding methodological rigor, sampling, handling of missing data, failure to deal with overfitting and heterogeneous definitions of COVID-19 and severity outcomes. While several clinical prognostic models for COVID-19 have been described in the literature, they are limited in generalizability and/or applicability due to deficiencies in addressing fundamental statistical and methodological concerns. Future large, multi-centric and well-designed prognostic prospective studies are needed to clarify remaining uncertainties.

A Systematic Review of Predictor Composition, Outcomes, Risk of Bias, and Validation of COVID-19 Prognostic Scores

“…Therefore, complementary approaches are needed. We identified a few reviews on COVID-19 predictions or scores, all focusing on different approaches and yielding a (slightly) different set of models, both overall and in terms of low ROB [ 8 , 9 , 44 , 45 ]. The 4C score [ 12 ], the PRIEST model [ 23 ] and the NEWS2 were repeatedly discussed as favorable prognostic tools.…”

Predicting Mortality Risk in Hospitalized COVID-19 Patients: An Early Model Utilizing Clinical Symptoms

“…Since the early stages of the pandemic, numerous studies have focused on developing prognostic models to stratify patients effectively and reduce the rates of severe disease progression and mortality. In a review conducted by C. The area under the receiver operating characteristic (ROC) curve for mortality prognostic models ranged from 0.49 to 0.99, with sensitivity ranging from 15.4-100% and speci city ranging from 10.9-98.7% [10].…”

Development and Validation of a Robust and Interpretable Early Triaging Support System for Patients Hospitalized With COVID-19: Predictive Algorithm Modeling and Interpretation Study

“…Many prognostic models for patients with COVID-19 severity and mortality have been proposed, yet most were reported unsuitable for clinical application by several systematic review studies [9][10][11]. Most models were either at a high or unclear risk of bias (Wynants et al [10]: 305 out of 310 studies, 98.4%; Buttia et al [11]: 312 out of 314 studies, 99.4%) such that their reported discriminative performances were deemed neither reliable nor generalizable [10,11].…”

“…Many prognostic models for patients with COVID-19 severity and mortality have been proposed, yet most were reported unsuitable for clinical application by several systematic review studies [9][10][11]. Most models were either at a high or unclear risk of bias (Wynants et al [10]: 305 out of 310 studies, 98.4%; Buttia et al [11]: 312 out of 314 studies, 99.4%) such that their reported discriminative performances were deemed neither reliable nor generalizable [10,11]. These high-risk models were developed with predictors selected based on univariable analysis, failed to deal with model overfitting represented by miscalibration, performed no or limited external validations with sufficient samples, imputed missing data without a clear explanation, or considered a limited number of machine learning (ML) algorithms [10,11].…”