Weight Excitation: Built-in Attention Mechanisms in Convolutional Neural Networks

Quader, Niamul; Bhuiyan, Mafijul Islam; Lu, Juwei; Dai, Peng; Li, Wei

doi:10.1007/978-3-030-58577-8_6

Cited by 23 publications

(18 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the WEB, a sub-block for location-based weight excitation (LWE) proposed in [31] is used. The LWE provides fine-grained weight-wise attention during back propagation.…”

Section: Position Attention-aware Weight Excitation (Pawe)mentioning

confidence: 99%

AWEU-Net: An Attention-Aware Weight Excitation U-Net for Lung Nodule Segmentation

et al. 2021

View full text Add to dashboard Cite

Lung cancer is a deadly cancer that causes millions of deaths every year around the world. Accurate lung nodule detection and segmentation in computed tomography (CT) images is a vital step for diagnosing lung cancer early. Most existing systems face several challenges, such as the heterogeneity in CT images and variation in nodule size, shape, and location, which limit their accuracy. In an attempt to handle these challenges, this article proposes a fully automated deep learning framework that consists of lung nodule detection and segmentation models. Our proposed system comprises two cascaded stages: (1) nodule detection based on fine-tuned Faster R-CNN to localize the nodules in CT images, and (2) nodule segmentation based on the U-Net architecture with two effective blocks, namely position attention-aware weight excitation (PAWE) and channel attention-aware weight excitation (CAWE), to enhance the ability to discriminate between nodule and non-nodule feature representations. The experimental results demonstrate that the proposed system yields a Dice score of 89.79% and 90.35%, and an intersection over union (IoU) of 82.34% and 83.21% on the publicly available LUNA16 and LIDC-IDRI datasets, respectively.

show abstract

“…In the WEB, a sub-block for location-based weight excitation (LWE) proposed in [31] is used. The LWE provides fine-grained weight-wise attention during back propagation.…”

Section: Position Attention-aware Weight Excitation (Pawe)mentioning

confidence: 99%

AWEU-Net: An Attention-Aware Weight Excitation U-Net for Lung Nodule Segmentation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The proposed WEU-Net is a encoder-decoder model shown in Figure 1 that modified version of the popular U-Net [5] model for the biomedical image segmentation task. The encoder part of the WEU-Net is composed by weight excitation based CNN (WE-CNN) [6], ReLU activation and MaxPooling layers. The input size of the model is 256x256x1, because of the single channel CT lung nodule image.…”

Section: Proposed Modelmentioning

confidence: 99%

“…Hence, it is necessary to develop advanced architectures that can deal with the weaknesses of the previous architectures. In this paper, a weight excitation (WE) mechanism through the weight reparameterization-based backpropagation corrections is used from [6] and implemented with the U-Net encoder and decoder architecture to cope with the heterogeneity of lung nodule feature extraction efficiently, which is suitable for the segmentation of various forms of lung nodules.The main contributions of this research can be summarized as follows:…”

Section: Introductionmentioning

confidence: 99%

WEU-Net: A Weight Excitation U-Net for Lung Nodule Segmentation

Banu

Sarker

Abdel-Nasser

et al. 2021

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Lung cancer is a dangerous non-communicable disease attacking both women and men and every year it causes thousands of deaths worldwide. Accurate lung nodule segmentation in computed tomography (CT) images can help detect lung cancer early. Since there are different locations and indistinguishable shapes of lung nodules in CT images, the accuracy of the existing automated lung nodule segmentation methods still needs further enhancements. In an attempt towards overcoming the above-mentioned challenges, this paper presents WEU-Net; an end-to-end encoder-decoder deep learning approach to accurately segment lung nodules in CT images. Specifically, we use a U-Net network as a baseline and propose a weight excitation (WE) mechanism to encourage the deep learning network to learn lung nodule-relevant contextual features during the training stage. WEU-Net was trained and validated on a publicly available CT images dataset called LIDC-IDRI. The experimental results demonstrated that WEU-Net achieved a Dice score of 82.83% and a Jaccard similarity coefficient of 70.55%.

show abstract

“…After years of development, deep learning methods have achieved remarkable success on visual classification tasks [17,39,21,32]. The outstanding performance, however, heavily relies on large-scale labeled datasets [5].…”

Section: Introductionmentioning

confidence: 99%

Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencoder

Liang¹,

Zhang²,

Dai³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

State of the art (SOTA) few-shot learning (FSL) methods suffer significant performance drop in the presence of domain differences between source and target datasets. The strong discrimination ability on the source dataset does not necessarily translate to high classification accuracy on the target dataset. In this work, we address this cross-domain few-shot learning (CDFSL) problem by boosting the generalization capability of the model. Specifically, we teach the model to capture broader variations of the feature distributions with a novel noise-enhanced supervised autoencoder (NSAE). NSAE trains the model by jointly reconstructing inputs and predicting the labels of inputs as well as their reconstructed pairs. Theoretical analysis based on intra-class correlation (ICC) shows that the feature embeddings learned from NSAE have stronger discrimination and generalization abilities in the target domain. We also take advantage of NSAE structure and propose a two-step fine-tuning procedure that achieves better adaption and improves classification performance in the target domain. Extensive experiments and ablation studies are conducted to demonstrate the effectiveness of the proposed method. Experimental results show that our proposed method consistently outperforms SOTA methods under various conditions.

show abstract

Weight Excitation: Built-in Attention Mechanisms in Convolutional Neural Networks

Cited by 23 publications

References 17 publications

AWEU-Net: An Attention-Aware Weight Excitation U-Net for Lung Nodule Segmentation

AWEU-Net: An Attention-Aware Weight Excitation U-Net for Lung Nodule Segmentation

WEU-Net: A Weight Excitation U-Net for Lung Nodule Segmentation

Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencoder

Contact Info

Product

Resources

About