Multi-scale Context Intertwining for Semantic Segmentation

Self-attention mechanism has been widely used for various tasks. It is designed to compute the representation of each position by a weighted sum of the features at all positions. Thus, it can capture long-range relations for computer vision tasks. However, it is computationally consuming. Since the attention maps are computed w.r.t all other positions. In this paper, we formulate the attention mechanism into an expectation-maximization manner and iteratively estimate a much more compact set of bases upon which the attention maps are computed. By a weighted summation upon these bases, the resulting representation is low-rank and deprecates noisy information from the input. The proposed Expectation-Maximization Attention (EMA) module is robust to the variance of input and is also friendly in memory and computation. Moreover, we set up the bases maintenance and normalization methods to stabilize its training procedure. We conduct extensive experiments on popular semantic segmentation benchmarks including PAS-CAL VOC, PASCAL Context and COCO Stuff, on which we set new records 1 .

show abstract

“…Backbone mIoU (%) PSPNet [37] ResNet-101 47.8 DANet [11] ResNet-50 50.1 MSCI [20] ResNet-152 50.3 EMANet…”

Section: Methodsmentioning

confidence: 99%

Expectation-Maximization Attention Networks for Semantic Segmentation

Li¹,

Zhong

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

568

300

View full text Add to dashboard Cite

show abstract

“…PSP-Net [30] designs a pyramid pooling module to collect the effective contextual prior, containing information of different scales. The encoder-decoder structures [?, 6,8,9] fuse mid-level and high-level semantic features to obtain different scale context. Second, learning contextual dependencies over local features also contribute to feature representations.…”

Section: Related Workmentioning

confidence: 99%

Dual Attention Network for Scene Segmentation

Liu

Tian

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

5,414

2,777

View full text Add to dashboard Cite

In this paper, we address the scene segmentation task by capturing rich contextual dependencies based on the self-attention mechanism. Unlike previous works that capture contexts by multi-scale feature fusion, we propose a Dual Attention Network (DANet) to adaptively integrate local features with their global dependencies. Specifically, we append two types of attention modules on top of dilated FCN, which model the semantic interdependencies in spatial and channel dimensions respectively. The position attention module selectively aggregates the feature at each position by a weighted sum of the features at all positions. Similar features would be related to each other regardless of their distances. Meanwhile, the channel attention module selectively emphasizes interdependent channel maps by integrating associated features among all channel maps. We sum the outputs of the two attention modules to further improve feature representation which contributes to more precise segmentation results. We achieve new state-of-theart segmentation performance on three challenging scene segmentation datasets, i.e., Cityscapes, PASCAL Context and COCO Stuff dataset. In particular, a Mean IoU score of 81.5% on Cityscapes test set is achieved without using coarse data. 1 .

show abstract

“…We strive for extracting structure-aware features (N × C) from an input point set (N ×d). In neural image processing, skip-connection is a powerful tool to leverage features extracted across different layers of the network [16,20,21,35]. Following PointNet++ [42], most existing point-based networks extract multiple scales of information by hierarchically downsampling the input point sets [33,59].…”

Section: Feature Extraction Via Intra-level Dense Connectionsmentioning

confidence: 99%

Patch-Based Progressive 3D Point Set Upsampling

Wang

Huang

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Self Cite

243

185

View full text Add to dashboard Cite

Figure 1: We develop a deep neural network for 3D point set upsampling. Intuitively, our network learns different levels of detail in multiple steps, where each step focuses on a local patch from the output of the previous step. By progressively training our patch-based network end-to-end, we successfully upsample a sparse set of input points, step by step, to a dense point set with rich geometric details. Here we use circle plates for points rendering, which are color-coded by point normals. AbstractWe present a detail-driven deep neural network for point set upsampling. A high-resolution point set is essential for point-based rendering and surface reconstruction. Inspired by the recent success of neural image super-resolution techniques, we progressively train a cascade of patch-based upsampling networks on different levels of detail end-to-end. We propose a series of architectural design contributions that lead to a substantial performance boost. The effect of each technical contribution is demonstrated in an ablation study. Qualitative and quantitative experiments show that our method significantly outperforms the state-of-theart learning-based [58,59], and optimazation-based [23] approaches, both in terms of handling low-resolution inputs and revealing high-fidelity details. The data and code are at https://github.com/yifita/3pu.

show abstract

Multi-scale Context Intertwining for Semantic Segmentation

Cited by 162 publications

References 43 publications

Expectation-Maximization Attention Networks for Semantic Segmentation

Expectation-Maximization Attention Networks for Semantic Segmentation

Dual Attention Network for Scene Segmentation

Patch-Based Progressive 3D Point Set Upsampling

Contact Info

Product

Resources

About