CAM-UNET: Class Activation MAP Guided UNET with Feedback Refinement for Defect Segmentation

Lin, Dongyun; Li, Yiqun; Prasad, Shitala; Nwe, Tin Lay; Dong, Suh-Yeon; Oo, Zaw Min

doi:10.1109/icip40778.2020.9190900

Cited by 17 publications

(17 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The U-Net model has thus become the principal segmentation model architecture to be used for natural and medical image segmentation tasks. Several U-Net model variants have been proposed, including V-Net [ 21 ], improved attention U-Net [ 22 ], nnU-Net [ 23 ], and U-Nets using ImageNet-pretrained encoders [ 24 ].…”

Section: Introductionmentioning

confidence: 99%

“…These models use the CXR modality-specific pretrained VGG-16 and VGG-19 models from the previous step as the encoder backbone. The performance of these models and other SOTA U-Net model variants, including the standard U-Net [ 20 ], V-Net with ResNet blocks [ 21 ], improved attention U-Net [ 22 ], ImageNet-pretrained VGG-16 U-Net [ 24 ], and ImageNet-pretrained VGG-19-U-Net are evaluated toward the lung segmentation task. The best performing model is used to segment lungs in a combined selection of CXRs showing normal lungs or pulmonary TB manifestations;…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improved Semantic Segmentation of Tuberculosis—Consistent Findings in Chest X-rays Using Augmented Training of Modality-Specific U-Net Models with Weak Localizations

et al. 2021

View full text Add to dashboard Cite

Deep learning (DL) has drawn tremendous attention for object localization and recognition in both natural and medical images. U-Net segmentation models have demonstrated superior performance compared to conventional hand-crafted feature-based methods. Medical image modality-specific DL models are better at transferring domain knowledge to a relevant target task than those pretrained on stock photography images. This character helps improve model adaptation, generalization, and class-specific region of interest (ROI) localization. In this study, we train chest X-ray (CXR) modality-specific U-Nets and other state-of-the-art U-Net models for semantic segmentation of tuberculosis (TB)-consistent findings. Automated segmentation of such manifestations could help radiologists reduce errors and supplement decision-making while improving patient care and productivity. Our approach uses the publicly available TBX11K CXR dataset with weak TB annotations, typically provided as bounding boxes, to train a set of U-Net models. Next, we improve the results by augmenting the training data with weak localization, postprocessed into an ROI mask, from a DL classifier trained to classify CXRs as showing normal lungs or suspected TB manifestations. Test data are individually derived from the TBX11K CXR training distribution and other cross-institutional collections, including the Shenzhen TB and Montgomery TB CXR datasets. We observe that our augmented training strategy helped the CXR modality-specific U-Net models achieve superior performance with test data derived from the TBX11K CXR training distribution and cross-institutional collections (p < 0.05). We believe that this is the first study to i) use CXR modality-specific U-Nets for semantic segmentation of TB-consistent ROIs and ii) evaluate the segmentation performance while augmenting the training data with weak TB-consistent localizations.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Improved Semantic Segmentation of Tuberculosis—Consistent Findings in Chest X-rays Using Augmented Training of Modality-Specific U-Net Models with Weak Localizations

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Ref. [4] proposed a class activation map guided U-Net with feedback refinement (CAM-UNet) for defect segmentation. It exploits both normal and anomalous images to train a classification network during its pretraining stage and generates the CAM of anomalous class as the prior segmentation of defective regions.…”

Section: Introductionmentioning

confidence: 99%

“…Here, we further modified CAM-UNet [4] by incorporating the normal images to regularize the training of the segmentation network at the fine-tuning stage. The regularization is motivated based on that the encoding network of the segmentation network should generate consistent representations of normal images and that of the normal regions within anomalous images.…”

Section: Introductionmentioning

confidence: 99%

“…The encoding network is then trained to "fool" the discriminator by generating the consistent representations of normal components from input images. By incorporating this adversarial regularization into the fine-tuning stage, our CAM-UNet-AR can produce better defect segmentation performance than CAM-UNet [4]. The major contributions of this paper can be summarized as follows:…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cam-Guided U-Net With Adversarial Regularization For Defect Segmentation

Lin

Prasad

et al. 2021

2021 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Defect segmentation is critical in real-wold industrial product quality assessment. There are usually a huge number of normal (defect-free) images but a very limited number of annotated anomalous images. This poses huge challenges to exploiting Fully-Convolutional Networks (FCN), e.g., UNet, as they require sufficient anomalous images with defect annotations during training. To further leverage the information from normal data, a novel CAM-guided U-Net with adversarial regularization (CAM-UNet-AR) is proposed. We first modify the existing CAM-UNet to incorporate the CAMs for both normal and anomalous classes and fine-tune the segmentation network using a combined loss which jointly considers pixel-wise classification, foreground segmentation and boundary segmentation. Secondly, an auxiliary adversarial regularization module (ARM) is proposed to facilitate the segmentation network to encode the "normal components" from training images into consistent representations. Extensive experiments on MVTec AD dataset show the superiority of our proposed network over multiple state-of-the-art U-Net variants.

show abstract

A neural ordinary differential equation model for visualizing deep neural network behaviors in multi‐parametric MRI‐based glioma segmentation

Yang

et al. 2023

Medical Physics

View full text Add to dashboard Cite

PurposeTo develop a neural ordinary differential equation (ODE) model for visualizing deep neural network behavior during multi‐parametric MRI‐based glioma segmentation as a method to enhance deep learning explainability.MethodsBy hypothesizing that deep feature extraction can be modeled as a spatiotemporally continuous process, we implemented a novel deep learning model, Neural ODE, in which deep feature extraction was governed by an ODE parameterized by a neural network. The dynamics of (1) MR images after interactions with the deep neural network and (2) segmentation formation can thus be visualized after solving the ODE. An accumulative contribution curve (ACC) was designed to quantitatively evaluate each MR image's utilization by the deep neural network toward the final segmentation results.The proposed Neural ODE model was demonstrated using 369 glioma patients with a 4‐modality multi‐parametric MRI protocol: T1, contrast‐enhanced T1 (T1‐Ce), T2, and FLAIR. Three Neural ODE models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. The key MRI modalities with significant utilization by deep neural networks were identified based on ACC analysis. Segmentation results by deep neural networks using only the key MRI modalities were compared to those using all four MRI modalities in terms of Dice coefficient, accuracy, sensitivity, and specificity.ResultsAll Neural ODE models successfully illustrated image dynamics as expected. ACC analysis identified T1‐Ce as the only key modality in ET and TC segmentations, while both FLAIR and T2 were key modalities in WT segmentation. Compared to the U‐Net results using all four MRI modalities, the Dice coefficient of ET (0.784→0.775), TC (0.760→0.758), and WT (0.841→0.837) using the key modalities only had minimal differences without significance. Accuracy, sensitivity, and specificity results demonstrated the same patterns.ConclusionThe Neural ODE model offers a new tool for optimizing the deep learning model inputs with enhanced explainability. The presented methodology can be generalized to other medical image‐related deep‐learning applications.

show abstract

CAM-UNET: Class Activation MAP Guided UNET with Feedback Refinement for Defect Segmentation

Cited by 17 publications

References 12 publications

Improved Semantic Segmentation of Tuberculosis—Consistent Findings in Chest X-rays Using Augmented Training of Modality-Specific U-Net Models with Weak Localizations

Improved Semantic Segmentation of Tuberculosis—Consistent Findings in Chest X-rays Using Augmented Training of Modality-Specific U-Net Models with Weak Localizations

Cam-Guided U-Net With Adversarial Regularization For Defect Segmentation

A neural ordinary differential equation model for visualizing deep neural network behaviors in multi‐parametric MRI‐based glioma segmentation

Contact Info

Product

Resources

About