Nonlocal spatial attention module for image classification

Abstract

To enhance the capability of neural networks, research on attention mechanism have been deepened. In this area, attention modules make forward inference along channel dimension and spatial dimension sequentially, parallelly, or simultaneously. However, we have found that spatial attention modules mainly apply convolution layers to generate attention maps, which aggregate feature responses only based on local receptive fields. In this article, we take advantage of this finding to create a nonlocal spatial attention module (NL-SAM), which collects context information from all pixels to adaptively recalibrate spatial responses in a convolutional feature map. NL-SAM overcomes the limitations of repeating local operations and exports a 2D spatial attention map to emphasize or suppress responses in different locations. Experiments on three benchmark datasets show at least 0.58% improvements on variant ResNets. Furthermore, this module is simple and can be easily integrated with existing channel attention modules, such as squeeze-and-excitation and gather-excite, to exceed these significant models at a minimal additional computational cost (0.196%).

Semantic Segmentation of Plant Leaves Based on Generative Adversarial Network and Attention Mechanism

“…A. Network structure SAM [28] (Spatial Attention Module): After the maximum and average pooling of the input convolution block, the position of the entire feature area can be determined through the convolution operation, and the influence of noise and rotation on the image can be filtered out. The definition of SAM is as follows:…”

Attention-based CNNs for Image Classification: A Survey

“…That is, the attention method uses obtained features of different parts of a network as weights to act other parts for learning more substantial sequential information. Current attention methods can be divided into two kinds: channel attention [17] and spatial attention methods [8]. Specifically, channel attention method emphasizes effects of channel features on the whole CNN.…”

“…Inspired by that, scholars combined an attention mechanism into a residual network [18]. That is divided into five kinds: single-path attention [18], multi-path attention [19], channel attention [17], spatial attention [8] and the combination of channel attention and spatial attention [20]. Specifically, a single-path attention method mainly uses its [17] enhanced the effects of different channels to improve the classification results.…”

“…Specifically, a single-path attention method mainly uses its [17] enhanced the effects of different channels to improve the classification results. Spatial attention method [8] used mean and max pooling operations to extract useful information for image classification as illustrated in Fig. 1.…”

“…That is, it fused outputs of current layer and previous layers as input of next layer to enhance the memory ability in image classification, which improved the performance at no extra cost in image classification [7]. To reduce the computational costs, CNNs based attention are used in image classification [8]. There are less literatures to summary CNNs based attention for image classifications.…”