Fully Attentional Network for Semantic Segmentation

Abstract

Recent non-local self-attention methods have proven to be effective in capturing long-range dependencies for semantic segmentation. These methods usually form a similarity map of R^(CxC) (by compressing spatial dimensions) or R^(HWxHW) (by compressing channels) to describe the feature relations along either channel or spatial dimensions, where C is the number of channels, H and W are the spatial dimensions of the input feature map. However, such practices tend to condense feature dependencies along the other dimensions, hence causing attention missing, which might lead to inferior results for small/thin categories or inconsistent segmentation inside large objects. To address this problem, we propose a new approach, namely Fully Attentional Network (FLANet), to encode both spatial and channel attentions in a single similarity map while maintaining high computational efficiency. Specifically, for each channel map, our FLANet can harvest feature responses from all other channel maps, and the associated spatial positions as well, through a novel fully attentional module. Our new method has achieved state-of-the-art performance on three challenging semantic segmentation datasets, i.e., 83.6%, 46.99%, and 88.5% on the Cityscapes test set, the ADE20K validation set, and the PASCAL VOC test set, respectively.

Attention-guided and resource-saving modules for semantic segmentation

“…In addition, choices of the pooling size combination are investigated. It is observed in Table 4 that the pooling size combination (4,8,12) outperforms (1,2,4) and (2,4,8), which makes it our settlement in MFA.…”

“…EMANet 11 iterates a set of compact bases using expectation maximization algorithm to represent the whole image and runs attention calculation on this set of bases, so that the computational complexity can be reduced significantly. FLANet 12 encodes both channel and spatial attentions in one single attention map, which not only considers all the information but also reduces the computational consumption. Even though these modules have reduced the computing cost and have improved the segmentation performance, they ignore the fusion of multi-resolution features.…”

TACT: Text attention based CNN‐Transformer network for polyp segmentation

“…Inspired by the Fully Attentional Block proposed in the literature 15 and Progressive Sampling proposed in the literature, 29 the FAPS module is proposed, shown in Figures 2 and 3. The basic idea of this module is to save the feature response from the global background under the same spatial position of horizontal and vertical coordinates and then use the self‐attention mechanism to capture the fully attention similarity between the two channel maps and their spatial positions.…”

“…Similarly, in non‐local spatial attention, a situation arises in which the action between each channel dimension is missing. Based on this, the literature 15 proposed Fully Attention Block (FLA), which uses global contextual features to preserve spatial response features when computing the channel attention map, which enables full attention in a single attention and improves computational efficiency. First, the feature response of the global context is captured at each spatial location.…”

Feature Consistency Constraints-Based CNN for Landsat Land Cover Mapping

“…EVALUATION OF SEGMENTATION RESULTS(%).To test the generality of the proposed feature consistency constraints, experiments on different network structures are conducted. In this experiment, four network structures, namely FCN, DeepLab V3+, PSPNet, and FLANet[56] are employed…”