Using Adversarial Images to Assess the Robustness of Deep Learning Models Trained on Diagnostic Images in Oncology

Abstract

PURPOSE Deep learning (DL) models have rapidly become a popular and cost-effective tool for image classification within oncology. A major limitation of DL models is their vulnerability to adversarial images, manipulated input images designed to cause misclassifications by DL models. The purpose of the study is to investigate the robustness of DL models trained on diagnostic images using adversarial images and explore the utility of an iterative adversarial training approach to improve the robustness of DL models against adversarial images. METHODS We examined the impact of adversarial images on the classification accuracies of DL models trained to classify cancerous lesions across three common oncologic imaging modalities. The computed tomography (CT) model was trained to classify malignant lung nodules. The mammogram model was trained to classify malignant breast lesions. The magnetic resonance imaging (MRI) model was trained to classify brain metastases. RESULTS Oncologic images showed instability to small pixel-level changes. A pixel-level perturbation of 0.004 (for pixels normalized to the range between 0 and 1) resulted in most oncologic images to be misclassified (CT 25.6%, mammogram 23.9%, and MRI 6.4% accuracy). Adversarial training improved the stability and robustness of DL models trained on oncologic images compared with naive models ([CT 67.7% v 26.9%], mammogram [63.4% vs 27.7%], and MRI [87.2% vs 24.3%]). CONCLUSION DL models naively trained on oncologic images exhibited dramatic instability to small pixel-level changes resulting in substantial decreases in accuracy. Adversarial training techniques improved the stability and robustness of DL models to such pixel-level changes. Before clinical implementation, adversarial training should be considered to proposed DL models to improve overall performance and safety.

Effectiveness of Deep Learning Classifiers in Histopathological Diagnosis of Oral Squamous Cell Carcinoma by Pathologists

“…Comparing the VGG16 and ResNet50 CNN models, the VGG16 performed better on the present dataset and hyperparameters. The VGG16 is a CNN architecture that has been demonstrated to improve robustness depending on the environment of the model 13 , and this was also observed in this study.…”

3D Capsule Networks for Brain Image Segmentation

“…1,[3][4][5][6] Manual segmentation is timeconsuming and is prone to intra-and inter-observer variability. 7,8 With the advent of deep learning to automate various image analysis tasks, 9,10 there has been increasing enthusiasm to use deep learning for brain image autosegmentation. [11][12][13][14] UNets are among the most popular and successful deep learning auto-segmentation algorithms.…”

Effective deep learning for oral exfoliative cytology classification

“…VGG16 has a structure in which the "convolution layer/convolution layer/pooling layer" is repeated twice, and the "convolution layer/convolution layer/convolution layer/pooling layer" is repeated three times, followed by three fully connected layers. It was reported that VGG16 is a model that can be expected to further improve robustness in recent year 28 . Therefore, we selected VGG16 as the CNN model in this study.…”