Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19

Nabulsi, Zaid; Sellergren, Andrew; Jamshy, Shahar; Lau, Charles T.; Santos, Edward Rainier G; Kiraly, Atilla P.; Ye, Wenxing; Yang, Jie; Pilgrim, Rory; Kazemzadeh, Sahar; Jin, Yu; Kalidindi, Sreenivasa Raju; Etemadi, Mozziyar; Garcia-Vicente, Florencia; Melnick, David S.; Corrado, Greg S.; Peng, Lily; Eswaran, Krishnan; Tse, Daniel; Beladia, Neeral; Liu, Yun; Chen, Po-Hsuan Cameron; Shetty, Shravya

doi:10.1038/s41598-021-93967-2

Cited by 36 publications

(31 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although we did not compare the performance with other models in the literature, the AUCs from of our AI algorithm was similar to those reported for other models assessed with open access CXR datasets (https://nihcc.app.box.com/v/ChestXray-NIHCC/file/220660789610) (20). Apart from detection of radiographic findings assessed in our study with an AI algorithm, other studies have also assessed applications of AI for prioritizing interpretation of CXRs to expedite reporting of abnormal CXRs and specific findings (21). Baltruschat et.…”

Section: Discussionsupporting

confidence: 79%

Performance of a Chest Radiography AI Algorithm for Detection of Missed or Mislabeled Findings: A Multicenter Study

Kaviani¹,

Digumarthy²,

Bizzo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Purpose: We assessed if a CXR AI algorithm can detect missed or mislabeled chest radiographs (CXRs) findings in radiology reports. Methods: We queried multi-institutional radiology reports search database of 13-million reports to identify all CXRs reports with addendums from 1999-2021. Of the 3469 CXR reports with an addendum, a thoracic radiologist excluded reports where addendum was created for typographic errors, wrong report template, missing sections, or uninterpreted signoffs. The remaining reports with addendum (279 patients) with errors related to side-discrepancy or missed findings such as pulmonary nodules, consolidation, pleural effusions, pneumothorax, and rib fractures. All CXRs were processed with an AI algorithm. Descriptive statistics were performed to determine the sensitivity, specificity, and accuracy of AI to detect missed or mislabeled findings. Results: AI had high sensitivity (96%), specificity (100%), and accuracy (96%) for detecting all missed and mislabeled CXR findings. The corresponding finding-specific statistics for AI were nodules (96%, 100%, 96%), pneumothorax (84%, 100%, 85%), pleural effusion (100%, 17%, 67%), consolidation (98%, 100%, 98%), and rib fractures (87%, 100%, 94%). Conclusion: The CXR AI could accurately detect mislabeled and missed findings. Clinical Relevance: The CXR AI can reduce the frequency of errors in detection and side-labeling of radiographic findings.

show abstract

Section: Discussionsupporting

confidence: 79%

Performance of a Chest Radiography AI Algorithm for Detection of Missed or Mislabeled Findings: A Multicenter Study

Kaviani¹,

Digumarthy²,

Bizzo³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Apart from detection of radiographic findings assessed in our study with an AI algorithm, other studies have assessed applications of AI for prioritizing interpretation of CXRs in order to expedite reporting of abnormal CXRs and specific findings [ 26 ]. Baltruschat et al reported the use of AI-based worklist prioritization for a substantial reduction in reporting turnaround time for critical CXR findings [ 27 ]. Similar improvements in reporting time with worklist prioritization using AI have been reported for other body regions as well, such as head CTs for intracranial hemorrhage [ 28 , 29 ].…”

Section: Discussionmentioning

confidence: 99%

Performance of a Chest Radiography AI Algorithm for Detection of Missed or Mislabeled Findings: A Multicenter Study

et al. 2022

View full text Add to dashboard Cite

We assessed whether a CXR AI algorithm was able to detect missed or mislabeled chest radiograph (CXR) findings in radiology reports. Methods: We queried a multi-institutional radiology reports search database of 13 million reports to identify all CXR reports with addendums from 1999–2021. Of the 3469 CXR reports with an addendum, a thoracic radiologist excluded reports where addenda were created for typographic errors, wrong report template, missing sections, or uninterpreted signoffs. The remaining reports contained addenda (279 patients) with errors related to side-discrepancies or missed findings such as pulmonary nodules, consolidation, pleural effusions, pneumothorax, and rib fractures. All CXRs were processed with an AI algorithm. Descriptive statistics were performed to determine the sensitivity, specificity, and accuracy of the AI in detecting missed or mislabeled findings. Results: The AI had high sensitivity (96%), specificity (100%), and accuracy (96%) for detecting all missed and mislabeled CXR findings. The corresponding finding-specific statistics for the AI were nodules (96%, 100%, 96%), pneumothorax (84%, 100%, 85%), pleural effusion (100%, 17%, 67%), consolidation (98%, 100%, 98%), and rib fractures (87%, 100%, 94%). Conclusion: The CXR AI could accurately detect mislabeled and missed findings. Clinical Relevance: The CXR AI can reduce the frequency of errors in detection and side-labeling of radiographic findings.

show abstract

“…As a result, it is expected that many of the labels assigned are incorrect. In some other studies of these data, expert labels are solicited for 810 selected testing images from multiple experienced radiologists [25].…”

Section: Estimating and Correcting Prediction Errorsmentioning

confidence: 99%

Topological structure of complex predictions

Li¹,

Dey²,

Gleich³

2022

Preprint

View full text Add to dashboard Cite

Complex prediction models such as deep learning are the output from fitting machine learning, neural networks, or AI models to a set of training data. These are now standard tools in science. A key challenge with the current generation of models is that they are highly parameterized, which makes describing and interpreting the prediction strategies difficult. We use topological data analysis to transform these complex prediction models into pictures representing a topological view. The result is a map of the predictions that enables inspection. The methods scale up to large datasets across different domains and enable us to detect labeling errors in training data, understand generalization in image classification, and inspect predictions of likely pathogenic mutations in the BRCA1 gene.

show abstract

Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19

Cited by 36 publications

References 35 publications

Performance of a Chest Radiography AI Algorithm for Detection of Missed or Mislabeled Findings: A Multicenter Study

Performance of a Chest Radiography AI Algorithm for Detection of Missed or Mislabeled Findings: A Multicenter Study

Performance of a Chest Radiography AI Algorithm for Detection of Missed or Mislabeled Findings: A Multicenter Study

Topological structure of complex predictions

Contact Info

Product

Resources

About