Artificial Intelligence in Chest Radiography Reporting Accuracy

Abstract

ObjectivesChest radiographs (CXRs) are commonly performed in emergency units (EUs), but the interpretation requires radiology experience. We developed an artificial intelligence (AI) system (precommercial) that aims to mimic board-certified radiologists' (BCRs') performance and can therefore support non–radiology residents (NRRs) in clinical settings lacking 24/7 radiology coverage. We validated by quantifying the clinical value of our AI system for radiology residents (RRs) and EU-experienced NRRs in a clinically representative EU setting.Materials and MethodsA total of 563 EU CXRs were retrospectively assessed by 3 BCRs, 3 RRs, and 3 EU-experienced NRRs. Suspected pathologies (pleural effusion, pneumothorax, consolidations suspicious for pneumonia, lung lesions) were reported on a 5-step confidence scale (sum of 20,268 reported pathology suspicions [563 images × 9 readers × 4 pathologies]) separately by every involved reader. Board-certified radiologists' confidence scores were converted into 4 binary reference standards (RFSs) of different sensitivities. The RRs' and NRRs' performances were statistically compared with our AI system (trained on nonpublic data from different clinical sites) based on receiver operating characteristics (ROCs) and operating point metrics approximated to the maximum sum of sensitivity and specificity (Youden statistics).ResultsThe NRRs lose diagnostic accuracy to RRs with increasingly sensitive BCRs' RFSs for all considered pathologies. Based on our external validation data set, the AI system/NRRs' consensus mimicked the most sensitive BCRs' RFSs with areas under ROC of 0.940/0.837 (pneumothorax), 0.953/0.823 (pleural effusion), and 0.883/0.747 (lung lesions), which were comparable to experienced RRs and significantly overcomes EU-experienced NRRs' diagnostic performance. For consolidation detection, the AI system performed on the NRRs' consensus level (and overcomes each individual NRR) with an area under ROC of 0.847 referenced to the BCRs' most sensitive RFS.ConclusionsOur AI system matched RRs' performance, meanwhile significantly outperformed NRRs' diagnostic accuracy for most of considered CXR pathologies (pneumothorax, pleural effusion, and lung lesions) and therefore might serve as clinical decision support for NRRs.

Implementing Artificial Intelligence for Emergency Radiology Impacts Physicians' Knowledge and Perception

“…Generally, older and more experienced physicians were more skeptical, were less likely to use AI results, and saw fewer benefits. This might be because experienced readers often show less pronounced improvement with AI support, but in numerous studies, their improvement also showed to be significant 5,12–14 …”

AUCReshaping: Improved sensitivity at high-specificity

“…ROC curve). As one of the main goals of a CAD system, is to reduce the variability among readers in an assisted setting 12,13 , any improvement of the CAD system performance outside the effective interval, will not impact the reader's performance in an assisted setting. Here, the region of interest (ROI) is shown where the performance improvement is most desired.…”

“…Moreover, we first study the improvements from the AUCReshaping function on a dataset of Chest X-Rays (CXRs). With the acquisition of millions of CXR images, the ability to filter out normal images with high confidence saves radiologists, the burden of parsing hundreds of images daily 12,13,18 . This allows them to focus on critical patients, dramatically speeding up the diagnostic process, which is especially critical in pandemic-like situations.…”

Artificial Intelligence to Assess Tracheal Tubes and Central Venous Catheters in Chest Radiographs Using an Algorithmic Approach With Adjustable Positioning Definitions

“…In recent years, artificial intelligence (AI) has shown high potential in mimicking health care specialists' performance in medical image interpretation. [16][17][18][19][20] Nevertheless, and despite over 50 years of research in conventional computer-aided diagnosis, there is still a paucity of algorithms for catheter evaluation in chest radiography, especially with regard to CVC evaluation. 11 Here, we present the development and validation of an AI algorithm that automatically assesses the positioning of CVCs and TTs in CXRs relative to anatomical landmarks, which is supposed to prospectively become integrated in a more extensive, CXR-analyzing software package.…”

“…Nevertheless, and despite over 50 years of research in conventional computer-aided diagnosis, there is still a paucity of algorithms for catheter evaluation in chest radiography, especially with regard to CVC evaluation 11 . Here, we present the development and validation of an AI algorithm that automatically assesses the positioning of CVCs and TTs in CXRs relative to anatomical landmarks, which is supposed to prospectively become integrated in a more extensive, CXR-analyzing software package 16,21,22 . The algorithm uses a convolutional neural network (CNN) approach and has been trained on nonpublicly available, annotated CXRs from different clinical sites.…”