External validation of AI algorithms in breast radiology: the last healthcare security checkpoint?

Martín-Noguerol, Teodoro; Luna, Antonio

doi:10.21037/qims-20-1409

Cited by 5 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, in developing research protocols, it is crucial to evaluate and consider robustness, replicability, and external validation ( 23 ). This paper presents an AI method based on morphological features, which differs from previous studies that solely compare deep learning models with doctors’ diagnostic results ( 24 ).…”

Section: Discussionmentioning

confidence: 99%

Artificial intelligence-based classification of breast nodules: a quantitative morphological analysis of ultrasound images

Pan,

Shi,

Zhang

et al. 2024

Quant Imaging Med Surg

View full text Add to dashboard Cite

Background Accurate classification of breast nodules into benign and malignant types is critical for the successful treatment of breast cancer. Traditional methods rely on subjective interpretation, which can potentially lead to diagnostic errors. Artificial intelligence (AI)-based methods using the quantitative morphological analysis of ultrasound images have been explored for the automated and reliable classification of breast cancer. This study aimed to investigate the effectiveness of AI-based approaches for improving diagnostic accuracy and patient outcomes. Methods In this study, a quantitative analysis approach was adopted, with a focus on five critical features for evaluation: degree of boundary regularity, clarity of boundaries, echo intensity, and uniformity of echoes. Furthermore, the classification results were assessed using five machine learning methods: logistic regression (LR), support vector machine (SVM), decision tree (DT), naive Bayes, and K-nearest neighbor (KNN). Based on these assessments, a multifeature combined prediction model was established. Results We evaluated the performance of our classification model by quantifying various features of the ultrasound images and using the area under the receiver operating characteristic (ROC) curve (AUC). The moment of inertia achieved an AUC value of 0.793, while the variance and mean of breast nodule areas achieved AUC values of 0.725 and 0.772, respectively. The convexity and concavity achieved AUC values of 0.988 and 0.987, respectively. Additionally, we conducted a joint analysis of multiple features after normalization, achieving a recall value of 0.98, which surpasses most medical evaluation indexes on the market. To ensure experimental rigor, we conducted cross-validation experiments, which yielded no significant differences among the classifiers under 5-, 8-, and 10-fold cross-validation (P>0.05). Conclusions The quantitative analysis can accurately differentiate between benign and malignant breast nodules.

show abstract

Section: Discussionmentioning

confidence: 99%

Artificial intelligence-based classification of breast nodules: a quantitative morphological analysis of ultrasound images

Pan,

Shi,

Zhang

et al. 2024

Quant Imaging Med Surg

View full text Add to dashboard Cite

show abstract

“… 19 Some AI tools conducted deep learning with mammograms only, and demographic and clinical data were not used in training sessions, which might have decreased the sensitivity or specificity. 20 In clinical practice, before drawing a conclusion, a radiologist would conduct a comprehensive evaluation based on the characteristics of mammograms and other risk factors, such as age, past medical history, family history, and estrogen exposure, which was a missing step in the AI algorithm. The challenge for AI was the inclusion of a large number of variables in the training of the algorithm (e.g., age, prior breast cancer, hormone supply treatment) and imitating the radiologist's workflow, which demanded large‐ and high‐quality data.…”

Section: Discussionmentioning

confidence: 99%

“…AI techniques for breast imaging studies, such as artificial neural networks, 17 machine learning, 18 and deep learning, had varied 19 . Some AI tools conducted deep learning with mammograms only, and demographic and clinical data were not used in training sessions, which might have decreased the sensitivity or specificity 20 . In clinical practice, before drawing a conclusion, a radiologist would conduct a comprehensive evaluation based on the characteristics of mammograms and other risk factors, such as age, past medical history, family history, and estrogen exposure, which was a missing step in the AI algorithm.…”

Section: Discussionmentioning

confidence: 99%

Evaluation of an artificial intelligence support system for breast cancer screening in Chinese people based on mammogram

et al. 2022

View full text Add to dashboard Cite

Background To evaluate the diagnostic performance of radiologists on breast cancer with or without artificial intelligence (AI) support. Methods A retrospective study was performed. In total, 643 mammograms (average age: 54 years; female: 100%; cancer: 62.05%) were randomly allocated into two groups. Seventy‐five percent of mammograms in each group were randomly selected for assessment by two independent radiologists, and the rest were read once. Half of the 71 radiologists could read mammograms with AI support, and the other half could not. Sensitivity, specificity, Youden's index, agreement rate, Kappa value, the area under the receiver operating characteristic curve (AUC) and the reading time of radiologists in each group were analyzed. Results The average AUC was higher if the AI support system was used (unaided: 0.84; with AI support: 0.91; p < 0.01). The average sensitivity increased from 84.77% to 95.07% with AI support ( p < 0.01), but the average specificity decreased ( p = 0.07). Youden's index, agreement rate and Kappa value were larger in the group with AI support, and the average reading time was shorter ( p < 0.01). Conclusions The AI support system might contribute to enhancing the diagnostic performance (e.g., higher sensitivity and AUC) of radiologists. In the future, the AI algorithm should be improved, and prospective studies should be conducted.

show abstract

“…Moreover, it is noteworthy that there are already a few studies available that investigate security and robustness aspects of AI models for cancer detection. Approaches such as the external validation of AI algorithms [108] and robustness tests against adversarial images [109], as well as comprehensive data preprocessing [110,111], are promising to achieve robustness and security goals and should therefore be investigated in more detail. In this context, the application of design science research could also be a way to iteratively address specific security problems in order to find an efficient solution.…”

Section: Future Research Agendamentioning

confidence: 99%

Artificial Intelligence for Cancer Detection—A Bibliometric Analysis and Avenues for Future Research

Karger

Kureljusic

2023

Current Oncology

View full text Add to dashboard Cite

After cardiovascular diseases, cancer is responsible for the most deaths worldwide. Detecting a cancer disease early improves the chances for healing significantly. One group of technologies that is increasingly applied for detecting cancer is artificial intelligence. Artificial intelligence has great potential to support clinicians and medical practitioners as it allows for the early detection of carcinomas. During recent years, research on artificial intelligence for cancer detection grew a lot. Within this article, we conducted a bibliometric study of the existing research dealing with the application of artificial intelligence in cancer detection. We analyzed 6450 articles on that topic that were published between 1986 and 2022. By doing so, we were able to give an overview of this research field, including its key topics, relevant outlets, institutions, and articles. Based on our findings, we developed a future research agenda that can help to advance research on artificial intelligence for cancer detection. In summary, our study is intended to serve as a platform and foundation for researchers that are interested in the potential of artificial intelligence for detecting cancer.

show abstract

External validation of AI algorithms in breast radiology: the last healthcare security checkpoint?

Cited by 5 publications

References 24 publications

Artificial intelligence-based classification of breast nodules: a quantitative morphological analysis of ultrasound images

Artificial intelligence-based classification of breast nodules: a quantitative morphological analysis of ultrasound images

Evaluation of an artificial intelligence support system for breast cancer screening in Chinese people based on mammogram

Artificial Intelligence for Cancer Detection—A Bibliometric Analysis and Avenues for Future Research

Contact Info

Product

Resources

About