Development and external validation of prognostic models for COVID-19 to support risk stratification in secondary care

Abstract

ObjectivesExisting UK prognostic models for patients admitted to the hospital with COVID-19 are limited by reliance on comorbidities, which are under-recorded in secondary care, and lack of imaging data among the candidate predictors. Our aims were to develop and externally validate novel prognostic models for adverse outcomes (death and intensive therapy unit (ITU) admission) in UK secondary care and externally validate the existing 4C score.DesignCandidate predictors included demographic variables, symptoms, physiological measures, imaging and laboratory tests. Final models used logistic regression with stepwise selection.SettingModel development was performed in data from University Hospitals Birmingham (UHB). External validation was performed in the CovidCollab dataset.ParticipantsPatients with COVID-19 admitted to UHB January–August 2020 were included.Main outcome measuresDeath and ITU admission within 28 days of admission.Results1040 patients with COVID-19 were included in the derivation cohort; 288 (28%) died and 183 (18%) were admitted to ITU within 28 days of admission. Area under the receiver operating characteristic curve (AUROC) for mortality was 0.791 (95% CI 0.761 to 0.822) in UHB and 0.767 (95% CI 0.754 to 0.780) in CovidCollab; AUROC for ITU admission was 0.906 (95% CI 0.883 to 0.929) in UHB and 0.811 (95% CI 0.795 to 0.828) in CovidCollab. Models showed good calibration. Addition of comorbidities to candidate predictors did not improve model performance. AUROC for the International Severe Acute Respiratory and Emerging Infection Consortium 4C score in the UHB dataset was 0.753 (95% CI 0.720 to 0.785).ConclusionsThe novel prognostic models showed good discrimination and calibration in derivation and external validation datasets, and performed at least as well as the existing 4C score using only routinely collected patient information. The models can be integrated into electronic medical records systems to calculate each individual patient’s probability of death or ITU admission at the time of hospital admission. Implementation of the models and clinical utility should be evaluated.

External Validation of the Modified 4C Deterioration Model and 4C Mortality Score for COVID-19 Patients in a Swiss Tertiary Hospital

“…In line with other validation studies [20][21][22][23][24][25], the respective event rate in our cohort was lower than in the original studies. One reason might be that the patients in our cohort were younger than the original cohort.…”

External Validation of Mortality Scores among High-Risk COVID-19 Patients: A Romanian Retrospective Study in the First Pandemic Year

“…Adderley et al aimed to develop and externally validate novel prognostic models for adverse outcomes in the UK and externally validated the existing 4CM Score, adding more comorbidities, and the conclusion was that the new model’s performance was not significantly better when compared with the original score. They found an AUROC slightly higher than the value obtained in our study (0.753 (95% CI: 0.720 to 0.785)) [ 6 ].…”

“…A systematic review released in July 2020 found many published prognostic scores estimating the mortality risk in COVID-19 patients, with a high or unclear risk of bias, of which the 4CM Score was considered promising [ 5 ]. Other scores were recently proposed [ 6 , 7 , 8 , 9 ]. The worldwide applicability of these predictive scores remains an open question because healthcare systems and patient profiles differ between countries, and may impact the scores’ performance [ 10 , 11 , 12 , 13 , 14 , 15 ].…”

Oxidative stress markers-driven prognostic model to predict post-discharge mortality in heart failure with reduced ejection fraction

“…Furthermore, our model was developed based on the original statistical approach. The traditional statistical method based on logistic regression, commonly applied in previous predictive models for HFrEF, was explored ( 51 ). In addition, two novel machine learning approaches (random forest and extreme gradient boosting) were also applied to predict post-discharge mortality in patients with HFrEF.…”