Development and Validation of a Deep Learning–based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs

Abstract

Background Detection of active pulmonary tuberculosis on chest radiographs (CRs) is critical for the diagnosis and screening of tuberculosis. An automated system may help streamline the tuberculosis screening process and improve diagnostic performance. Methods We developed a deep learning–based automatic detection (DLAD) algorithm using 54c221 normal CRs and 6768 CRs with active pulmonary tuberculosis that were labeled and annotated by 13 board-certified radiologists. The performance of DLAD was validated using 6 external multicenter, multinational datasets. To compare the performances of DLAD with physicians, an observer performance test was conducted by 15 physicians including nonradiology physicians, board-certified radiologists, and thoracic radiologists. Image-wise classification and lesion-wise localization performances were measured using area under the receiver operating characteristic (ROC) curves and area under the alternative free-response ROC curves, respectively. Sensitivities and specificities of DLAD were calculated using 2 cutoffs (high sensitivity [98%] and high specificity [98%]) obtained through in-house validation. Results DLAD demonstrated classification performance of 0.977–1.000 and localization performance of 0.973–1.000. Sensitivities and specificities for classification were 94.3%–100% and 91.1%–100% using the high-sensitivity cutoff and 84.1%–99.0% and 99.1%–100% using the high-specificity cutoff. DLAD showed significantly higher performance in both classification (0.993 vs 0.746–0.971) and localization (0.993 vs 0.664–0.925) compared to all groups of physicians. Conclusions Our DLAD demonstrated excellent and consistent performance in the detection of active pulmonary tuberculosis on CR, outperforming physicians, including thoracic radiologists.

Deep Learning for Chest Radiograph Diagnosis in the Emergency Department

“…chest radiographs (19,20). Most of those studies evaluated the efficacy of these algorithms in enriched data sets, which differ from real-world findings in terms of disease prevalence, spectrum of presentation, and population diversity.…”

Clinical Implementation of Deep Learning in Thoracic Radiology: Potential Applications and Challenges

“…In addition to lung nodules, the DL-based algorithm has shown good performance in various thoracic diseases, such as pulmonary tuberculosis (area under receiver operating characteristic curve [AUC], 0.83-0.99) (28,29,44,45), pneumonia (maximum AUC in internal validation, 0.93) (46), and pneumothorax (AUC, 0.82-0.91) (32,47), and in the evaluation of medical devices on CXRs (48-50). However, algorithms specific to a single disease or abnormality may have limited value in real clinical practice, as the interpretation of CXR requires the assessment of various diseases and abnormalities in the thorax.…”

“…It is now anticipated that DL technology will overcome the limitations in performance shown by conventional computer-aided image analyses and be implemented in the daily practice of thoracic radiology (17,26). Indeed, there have been several early investigations reporting the surprisingly high performance of DL technologies in thoracic radiology, particularly CXRs (27)(28)(29)(30)(31)(32).…”

“…3) (28,29,44,45,62), suggesting a potential for implementation. Furthermore, in a study by Hwang et al (28), the DL algorithm (AUC, 0.99) outperformed human readers (AUC, 0.75-0.97), including thoracic radiologists and the performance of human readers improved after reviewing the algorithm results (AUC, 0.75-0.97 to 0.85-0.98). Thus far, DL algorithms, however, have been tested on datasets collected for case-control studies (45).…”

Clustered micronodules as predominant manifestation on CT: A sign of active but indolently evolving pulmonary tuberculosis

“…Artificial intelligence (AI) is a promising technique in medical fields, particularly for radiology. Indeed, the AI algorithm already achieved excellent performance in the detection of active tuberculosis on chest X-ray [19]. Detection of active tuberculosis on CT is still a challenging task, and CT achieved decent performance in the detection of a pulmonary nodule [20].…”