Nested attention network based on category contexts learning for semantic segmentation

Abstract

The attention mechanism is widely used in the field of semantic segmentation, due to the fact that it can be used to obtain effective long-distance dependencies by assigning different weights to objects according to different tasks. We propose a novel Nested Attention Network (NANet) for semantic segmentation, which combines Feature Category Attention (FCA) and Channel Relationship Attention (CRA) to effectively aggregate same-category contexts in both spatial and channel dimensions. Specifically, FCA captures the dependencies between spatial pixel features and categories to achieve the aggregation of features of the same category. CRA further captures the channel relationships on the output of FCA to obtain richer contexts. Numerous experiments have shown that NANet has a lower number of parameters and computational complexity than other state-of-the-art methods, and is a lightweight model with a lower total number of floating-point operations. We evaluated the performance of NANet on three datasets: Cityscapes, PASCAL VOC 2012, and ADE20K, and the experimental results show that NANet obtains promising results, reaching a performance of 82.6% on the Cityscapes test set.