Deep learning-based pulmonary tuberculosis automated detection on chest radiography: large-scale independent testing

Abstract

Background: It is critical to have a deep learning-based system validated on an external dataset before it is used to assist clinical prognoses. The aim of this study was to assess the performance of an artificial intelligence (AI) system to detect tuberculosis (TB) in a large-scale external dataset.Methods: An artificial, deep convolutional neural network (DCNN) was developed to differentiate TB from other common abnormalities of the lung on large-scale chest X-ray radiographs. An internal dataset with 7,025 images was used to develop the AI system, including images were from five sources in the U.S. and China, after which a 6-year dynamic cohort accumulation dataset with 358,169 images was used to conduct an independent external validation of the trained AI system.Results: The developed AI system provided a delineation of the boundaries of the lung region with a Dice coefficient of 0.958. It achieved an AUC of 0.99 and an accuracy of 0.948 on the internal data set, and an AUC of 0.95 and an accuracy of 0.931 on the external data set when it was used to detect TB from normal images. The AI system achieved an AUC of more than 0.9 on the internal data set, and an AUC of over 0.8 on the external data set when it was applied to detect TB, non-TB abnormal and normal images. Conclusions:We conducted a real-world independent validation, which showed that the trained system can be used as a TB screening tool to flag possible cases for rapid radiologic review and guide further examinations for radiologists.

Impact of localized fine tuning in the performance of segmentation and classification of lung nodules from computed tomography scans using deep learning

“…This may result from the significant appearance variances caused by the population and setting differences ( 32 – 34 ). It has been reported that involving multi-center datasets to develop algorithm is effective to keep the algorithm robust to maintain its accuracy across datasets ( 10 , 11 ). However, it is unclear how many datasets should be exactly included to create a robust detection algorithm to obtain comparable performances of the internal test, especially when those external datasets are significantly different from internal datasets.…”

“…It may halt the possible implementation of the general model into routine clinical care if it does not have a consistent accuracy for site-specific use. To obtain a comparable external test performance to the internal tests, reported studies involving training datasets from multicenter to develop the detection algorithm demonstrated that it can either underperform ( 10 – 12 ) or have a comparable performance to the internal test ( 11 , 13 ) without any unanimous conclusion reached, which may be explained by the differences of the datasets scale and the numbers of dataset origins ( 14 ). Using local images for model training seems to be another way to obtain a site-specific used tool for diagnosis.…”

Artificial Intelligence in Paediatric Tuberculosis

“…In recent years, AI and computer-aided detection software have been developed to augment and automate the interpretation of digital chest radiography in TB screening [ 7 ]. A literature search for publications including studies on AI in TB imaging using search items related to AI, TB and thoracic imaging yielded 110 results of which 21 were published in the last 5 years ( Table 1 ) [ 23 – 43 ]. It is important to note that all but 2 of the 21 articles [ 32 , 37 ] included studies where the algorithms were validated on data sets with a population age group > 15 years old.…”

“…A moderate to strong correlation was demonstrated between the AI model–quantified TB score and the radiologist-estimated CT score 20 Yi PH et al [ 42 ] Clinical Imaging (2022) Authors collected 10,951 CXRs from the NIH Chest X-ray 14 data set NIH Chest X-ray14 data set ResNet 50 Deep convolutional neural network (DCNN) training and validation done The best-performing deep convolutional neural network (DCNN) had an area under the curve (AUC) of 0.88, which was trained on 10,951 images using the radiologist-annotated sets. DCNNs trained on chest X-rays (CXRs) labelled by a radiologist consistently outperformed those trained on the same CXRs labelled by Natural Language Processing (NLP), highlighting the benefit of radiologists in determining ground truth for machine-learning data set curation 21 Zhou W et al [ 43 ] Quantitative Imaging in Medicine and Surgery (2022) An internal data set with 7,025 images was used to develop the AI system, after which a 6-year dynamic cohort accumulation data set with 358,169 images was used to conduct an independent external validation of the trained AI system Images were from five sources in the U.S. and China A transfer learning approach was applied to train a pretrained ResNet model to build the final DCNN to identify and locate TB. A U-Net-based algorithm was trained to automatically segment the lung area The AI system achieved an AUC of 0.99 and an accuracy of 0.948 on the internal data set, and an AUC of 0.95 and an accuracy of 0.931 on the external data set when it was used to detect TB from normal images A literature search for publications including studies on artificial intelligence (AI) in tuberculosis (TB) imaging using search items related to AI, TB and thoracic imaging yielded 110 results of which 21were published in the last 5 years a The algorithms used in these 2 studies were validated on data sets which included population age groups < 15 years old …”

Developing and assessing an AI-based multi-task prediction system to assist radiologists detecting lung diseases in reading chest x-ray images

“…In order to help address the clinical challenges and reduce diagnostic errors in reading and interpreting CXR images, developing computer-aided detection or diagnosis (CAD) schemes of CXR images using either the conventional medical image processing algorithms or the advanced artificial intelligence (AI) technologies has been attracting broad research interest in the last several decades. As a result, many groups of researchers have developed different CAD schemes to automatically detect pulmonary diseases such as tuberculosis, lung nodule and pneumonia using CXR images and reported decent performance compared to radiologists in the reported studies [7][8][9] . Several prospectively clinical evaluation studies have also been reported recently to assess potential clinical use of CAD schemes for CXR images 10 .…”