CCEDRRN COVID-19 Infection Score (CCIS): development and validation in a Canadian cohort of a clinical risk score to predict SARS-CoV-2 infection in patients presenting to the emergency department with suspected COVID-19

Abstract

ObjectivesTo develop and validate a clinical risk score that can accurately quantify the probability of SARS-CoV-2 infection in patients presenting to an emergency department without the need for laboratory testing.DesignCohort study of participants in the Canadian COVID-19 Emergency Department Rapid Response Network (CCEDRRN) registry. Regression models were fitted to predict a positive SARS-CoV-2 test result using clinical and demographic predictors, as well as an indicator of local SARS-CoV-2 incidence.Setting32 emergency departments in eight Canadian provinces.Participants27 665 consecutively enrolled patients who were tested for SARS-CoV-2 in participating emergency departments between 1 March and 30 October 2020.Main outcome measuresPositive SARS-CoV-2 nucleic acid test result within 14 days of an index emergency department encounter for suspected COVID-19 disease.ResultsWe derived a 10-item CCEDRRN COVID-19 Infection Score using data from 21 743 patients. This score included variables from history and physical examination and an indicator of local disease incidence. The score had a c-statistic of 0.838 with excellent calibration. We externally validated the rule in 5295 patients. The score maintained excellent discrimination and calibration and had superior performance compared with another previously published risk score. Score cut-offs were identified that can rule-in or rule-out SARS-CoV-2 infection without the need for nucleic acid testing with 97.4% sensitivity (95% CI 96.4 to 98.3) and 95.9% specificity (95% CI 95.5 to 96.0).ConclusionsThe CCEDRRN COVID-19 Infection Score uses clinical characteristics and publicly available indicators of disease incidence to quantify a patient’s probability of SARS-CoV-2 infection. The score can identify patients at sufficiently high risk of SARS-CoV-2 infection to warrant isolation and empirical therapy prior to test confirmation while also identifying patients at sufficiently low risk of infection that they may not need testing.Trial registration numberNCT04702945.

Evaluation of Diagnostic Strategies for Identifying SARS-CoV-2 Infection in Clinical Practice: a Systematic Review and Compliance with the Standards for Reporting Diagnostic Accuracy Studies Guideline (STARD)

“…Three of the articles that used AI ( 12 , 15 , 18 ) and four of the studies that developed an algorithm/probability score ( 19 , 20 , 26 , 27 ) validated the developed model in an independent sample. In addition, other studies internally validated the model through techniques such as the bootstrap technique (a statistical procedure that resamples a single dataset to create many simulated samples) ( 21 , 22 , 24 ) or a k-fold cross-validation process (an approach that randomly divides the set of observations into k groups, or folds, of approximately equal size; the first fold is treated as a validation set, and the method is fit on the remaining k-1 folds) ( 13 , 14 , 17 ) ( Table 1 ).…”

“…Table 1 shows the characteristics of the included studies, which have been classified according to the type of population included: (i) Patients with suspected SARS-CoV-2 infection (16, 69.6%) ( 12 – 27 ), (ii) Patients diagnosed with SARS-CoV-2 (6, 26.1%) ( 28 – 33 ), and (iii) other studies (1, 4.3%) ( 34 ).…”

“…Of the 16 studies which focused on symptomatic patients with suspected SARS-CoV-2 infection ( 12 – 27 ), seven included techniques related with artificial intelligence (AI) ( 12 – 18 ): five of them used a cohort study, either retrospective ( 12 , 13 , 17 , 18 ) or prospective ( 14 ), and two used a cross-sectional study ( 15 , 16 ). The other nine studies developed an algorithm ( 19 , 24 , 26 ) or a probability score ( 20 – 23 , 25 , 27 ) to diagnose infection by SARS-CoV-2. Four of them used a cross-sectional study ( 22 – 25 ), four used a retrospective cohort study ( 19 – 21 , 26 ), and one used a prospective cohort study ( 27 ).…”

“…The other nine studies developed an algorithm ( 19 , 24 , 26 ) or a probability score ( 20 – 23 , 25 , 27 ) to diagnose infection by SARS-CoV-2. Four of them used a cross-sectional study ( 22 – 25 ), four used a retrospective cohort study ( 19 – 21 , 26 ), and one used a prospective cohort study ( 27 ). Most of the studies included symptomatic patients who were admitted to a hospital due to suspected COVID-19; Trubiano JA et al’s study ( 24 ) included patients assessed for COVID-19 at a screening clinic due to symptoms (98.3%), contact with known COVID-19-positive patient (17.3%), or those who had traveled abroad (24.6%), and Elimian KO et al ( 26 ) included symptomatic patients and also those concerned about COVID-19 infection.…”

Reducing barriers to accessing administrative data on SARS-CoV-2 vaccination for research

“…The network has used these data to develop risk prediction tools to estimate the probability that a symptomatic patient in the emergency department would test positive for SARS-CoV-2, and the probability that a patient with COVID-19 would have an adverse outcome. [16][17][18] The network has funding from the Canadian Immunization Task Force and the Canadian Institutes of Health Research to conduct postmarketing studies of vaccine effectiveness (including in populations who were excluded from premarketing randomized trials), to identify risk factors for development of post-COVID-19 conditions and to update existing risk scores to account for vaccination status.…”