The accuracy of teleradiologists in diagnosing COVID-19 based on a French multicentric emergency cohort

Abstract

Objectives To evaluate the accuracy of diagnoses of COVID-19 based on chest CT as well as inter-observer agreement between teleradiologists during on-call duty and senior radiologists in suspected COVID-19 patients. Materials and methods From March 13, 2020, to April 14, 2020, consecutive suspected COVID-19 adult patients who underwent both an RT-PCR test and chest CT from 15 hospitals were included in this prospective study. Chest CTs were immediately interpreted by the on-call teleradiologist and were systematically blind reviewed by a senior radiologist. Readings were categorised using a five-point scale: (1) normal; (2) non-infectious findings; (3) infectious findings but not consistent with COVID-19 infection; (4) consistent with COVID-19 infection; and (5) typical appearance of COVID-19 infection. The diagnostic accuracy of chest CT and inter-observer agreement using the kappa coefficient were evaluated over the study period. Results In total, 513 patients were enrolled, of whom 244/513 (47.6%) tested positive for RT-PCR. First readings were scored 4 or 5 in 225/244 (92%) RT-PCR+ patients, and between 1 and 3 in 201/269 (74.7%) RT-PCR− patients. The data were highly consistent (weighted kappa = 0.87) and correlated with RT-PCR ( p < 0.001, AUC 1st-reading = 0.89, AUC 2nd-reading = 0.93). The negative predictive value for scores of 4 or 5 was 0.91–0.92, and the PPV for a score of 5 was 0.89–0.96 at the first and second readings, respectively. Diagnostic accuracy was consistent over the study period, irrespective of a variable prevalence rate. Conclusion Chest CT demonstrated high diagnostic accuracy with strong inter-observer agreement between on-call teleradiologists with varying degrees of experience and senior radiologists over the study period. Key Points • The accuracy of readings by on-call teleradiologists, relative to second readings by senior radiologists, demonstrated a sensitivity of 0.75–0.79, specificity of 0.92–0.97, NPV of 0.80–0.83, and PPV of 0.89–0.96, based on “typical appearance,” as predictive of RT-PCR+. • Inter-observer agreement between the first reading in the emergency setting and the second reading by the senior emergency teleradiologist was excellent (weighted kappa = 0.87). Electronic supplementary material The online version of this article (10.1007/s00330-020-07345-z) contains supplementary material, which is available to authorized users.

Telehealth-Based Services During the COVID-19 Pandemic: A Systematic Review of Features and Challenges

“…Regarding diagnosis purpose, in some studies, teleconsultation services were used by physicians to do diagnosis procedures. In some other studies, an attempt was made to perform the diagnosis process with the patients' participation accurately ( 7 , 42 , 60 , 62 ).…”

Emergency teleradiological activity is an epidemiological estimator and predictor of the covid-19 pandemic in mainland France

“…Images are centralised in three interpretation centres where teams of radiologists use the same tools to interpret images and report findings to emergency physicians. Moreover, resource organisation and structured processes, particularly chest CT, provide high diagnostic accuracy with strong inter-observer agreement between on-call teleradiologists with varying degrees of experience and senior radiologists [4]. Indeed, thanks to our structured organisation, we already showed that teleradiology enabled us to monitor the impact of the COVID-19 pandemic management on emergency activities, showing a global decrease in the population's use of care during the first lockdown in France [5].…”

“…We would have had to exclude all the observations from the first wave. However, we believe that using the French Society of Radiology scoring system could provide more powerful predictors in future radiological forecasting models-although strongly suggestive/compatible conclusions with COVID-19 diagnosis do not systematically mean positive RT-PCR despite the excellent accuracy of the French Society of Radiology scoring system [4]. Finally, it is important to consider that existing indicators based on hospitalisations should become less accurate with vaccination, necessitating original indicators.…”

Practical clinical and radiological models to diagnose COVID-19 based on a multicentric teleradiological emergency chest CT cohort

“…have demonstrated that IMADIS structuration enables to organize systematic second readings of whole body CT scanners in multiple traumas patients, with good diagnostic performances of OCTRs whatever the amount of activity and the hours of the day 32 . Regarding the OCTRs’ accuracy for diagnosing COVID-19, Nivet et al have also shown that implementing the semi-quantitative assessment as proposed by SFR was feasible and provided high sensitivity (0.92), specificity (0.75–0.84), PPV (0.77–0.84) and NPV (0.91–0.92) depending on the reading and when considering the indeterminate/compatible group as positive 12 .…”

“…Indeed, our population represents a real-life emergency cohort and some patients could have had non-infectious diseases (: conclusion (2)), or non-pulmonary infections for instance (: conclusion (1)), in addition to non-COVID-19 lung infectious diseases (: conclusion (3)). Finally, although widely used and significantly reproducible over multiple raters, the radiological features were qualitatively or semi-quantitatively assessed, which introduces a risk of subjectivity and could bias the performances of the model in other validation cohorts 12 .…”