Stage-Aware Feature Alignment Network for Real-Time Semantic Segmentation of Street Scenes

Weng, Xi; Yan, Yan; Chen, Si; Xue, Jing-Hao; Wang, Hanzi

doi:10.1109/tcsvt.2021.3121680

Cited by 43 publications

(14 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Huang et al [10] proposed the criss-cross attention to capture contextual information in an efficient way. Similar to [10], Liu et al [46] and Weng et al [15] focused on the efficiency of algorithms, and developed real-time and accurate segmentation methods. To better mine global contextual dependencies, Strudel et al [11] built the Segmenter on ViT [47].…”

Section: Related Work a Rgb Semantic Segmentationmentioning

confidence: 99%

RGB-T Semantic Segmentation With Location, Activation, and Sharpening

Wang

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Semantic segmentation is important for scene understanding. To address the scenes of adverse illumination conditions of natural images, thermal infrared (TIR) images are introduced. Most existing RGB-T semantic segmentation methods follow three cross-modal fusion paradigms, i.e., encoder fusion, decoder fusion, and feature fusion. Some methods, unfortunately, ignore the properties of RGB and TIR features or the properties of features at different levels. In this paper, we propose a novel feature fusion-based network for RGB-T semantic segmentation, named LASNet, which follows three steps of location, activation, and sharpening. The highlight of LASNet is that we fully consider the characteristics of cross-modal features at different levels, and accordingly propose three specific modules for better segmentation. Concretely, we propose a Collaborative Location Module (CLM) for high-level semantic features, aiming to locate all potential objects. We propose a Complementary Activation Module for middle-level features, aiming to activate exact regions of different objects. We propose an Edge Sharpening Module (ESM) for lowlevel texture features, aiming to sharpen the edges of objects. Furthermore, in the training phase, we attach a location supervision and an edge supervision after CLM and ESM, respectively, and impose two semantic supervisions in the decoder part to facilitate network convergence. Experimental results on two public datasets demonstrate that the superiority of our LASNet over relevant state-of-the-art methods. The code and results of our method are available at https://github.com/MathLee/LASNet.

show abstract

Section: Related Work a Rgb Semantic Segmentationmentioning

confidence: 99%

RGB-T Semantic Segmentation With Location, Activation, and Sharpening

Wang

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…DeepLab [21] employs dilated convolution operations to enlarge receptive fields for non-local semantical context. SFANet [4] alleviates the misalignment gap and seeks the balance between accuracy and inference speed. Tian et al [26] dealt with semantic segmentation in the framework of unsupervised domain adaptation and proposed partial domain adaptation to avoid the negative transfer problem.…”

Section: B Edge-enhanced Semantic Segmentationmentioning

confidence: 99%

“…SED can be viewed as a dual-task of semantic segmentation. Differently, semantic segmentation decomposes an image into semantic regions by categorizing each pixel according to its spatial context [4]. The obtained semantic regions can be converted to semantic edges by keeping only boundary points and their semantical categories [5].…”

Section: Introductionmentioning

confidence: 99%

All-Higher-Stages-In Adaptive Context Aggregation for Semantic Edge Detection

Lai

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) can reveal local variation details and multi-scale spatial context in images via low-to-high stages of feature expression; effective fusion of these raw features is key to Semantic Edge Detection (SED). The methods available in the field generally fuse features across stages in a position-aligned mode, which cannot satisfy the requirements of diverse semantic context in categorizing different pixels. In this paper, we propose a deep framework for SED, the core of which is a new multi-stage feature fusion structure, called All-HiS-In ACA (All-Higher-Stages-In Adaptive Context Aggregation).All-HiS-In ACA can adaptively select semantic context from all higher-stages for detailed features via a cross-stage selfattention paradigm, and thus can obtain fused features with highresolution details for edge localization and rich semantics for edge categorization. In addition, we develop a non-parametric Interlayer Complementary Enhancement (ICE) module to supplement clues at each stage with their counterparts in adjacent stages. The ICE-enhanced multi-stage features are then fed into the All-HiS-In ACA module. We also construct an Object-level Semantic Integration (OSI) module to further refine the fused features by enforcing the consistency of the features within the same object. Extensive experiments demonstrate the superior performance of the proposed method over state-of-the-art works.

show abstract

“…T HE task of single image reflection removal (SIRR) is to recover a clear transmission image by removing reflection from the blended image. This task is of significant importance in computational photography, as it not only enhances image quality but also has a positive impact on downstream computer vision tasks, such as object detection [1]- [3] and semantic segmentation [4], [5]. Since the reflection removal problem is Z. Zhang, Z.…”

Section: Introductionmentioning

confidence: 99%

Benchmarking Ultra-High-Definition Image Super-resolution

Luo

Ren³

et al. 2021

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Deep learning based methods have achieved significant success in the task of single image reflection removal (SIRR). However, the majority of these methods are focused on High-Definition/Standard-Definition (HD/SD) images, while ignoring higher resolution images such as Ultra-High-Definition (UHD) images. With the increasing prevalence of UHD images captured by modern devices, in this paper, we aim to address the problem of UHD SIRR. Specifically, we first synthesize two large-scale UHD datasets, UHDRR4K and UHDRR8K. The UHDRR4K dataset consists of 2, 999 and 168 quadruplets of images for training and testing respectively, and the UHDRR8K dataset contains 1, 014 and 105 quadruplets. To the best of our knowledge, these two datasets are the first largest-scale UHD datasets for SIRR. Then, we conduct a comprehensive evaluation of six state-of-the-art SIRR methods using the proposed datasets. Based on the results, we provide detailed discussions regarding the strengths and limitations of these methods when applied to UHD images. Finally, we present a transformerbased architecture named RRFormer for reflection removal. RRFormer comprises three modules, namely the Prepossessing Embedding Module, Self-attention Feature Extraction Module, and Multi-scale Spatial Feature Extraction Module. These modules extract hypercolumn features, global and partial attention features, and multi-scale spatial features, respectively. To ensure effective training, we utilize three terms in our loss function: pixel loss, feature loss, and adversarial loss. We demonstrate through experimental results that RRFormer achieves stateof-the-art performance on both the non-UHD dataset and our proposed UHDRR datasets. The code and datasets are publicly available at https://github.com/Liar-zzy/Benchmarking-Ultra-High-Definition-Single-Image-Reflection-Removal.

show abstract

Stage-Aware Feature Alignment Network for Real-Time Semantic Segmentation of Street Scenes

Cited by 43 publications

References 56 publications

RGB-T Semantic Segmentation With Location, Activation, and Sharpening

RGB-T Semantic Segmentation With Location, Activation, and Sharpening

All-Higher-Stages-In Adaptive Context Aggregation for Semantic Edge Detection

Benchmarking Ultra-High-Definition Image Super-resolution

Contact Info

Product

Resources

About