DSANet: Dilated spatial attention for real-time semantic segmentation in urban street scenes

Elhassan, Mohammed A. M.; Huang, Chenxi; Yang, Chenhui; Munea, Tewodros Legesse

doi:10.1016/j.eswa.2021.115090

Cited by 79 publications

(18 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various kinds of deep learning models [4,5,11,33,40] have been proposed to resolve visual applications with multi-task learning setups such as semantic segmentation, and depth estimation, efficiently. Khattar et al [16] propose a multi-task learning framework where domain-agnostic features are learned to improve the model performance on both object detection and saliency prediction tasks with limited data.…”

Section: Motivation and Challengesmentioning

confidence: 99%

MetaMorphosis: Task-oriented Privacy Cognizant Feature Generation for Multi-task Learning

Arefeen

Uddin

et al. 2023

Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation

View full text Add to dashboard Cite

With the growth of computer vision applications, deep learning, and edge computing contribute to ensuring practical collaborative intelligence (CI) by distributing the workload among edge devices and the cloud. However, running separate single-task models on edge devices is inefficient regarding the required computational resource and time. In this context, multitask learning allows leveraging a single deep learning model for performing multiple tasks, such as semantic segmentation and depth estimation on incoming video frames. This single processing pipeline generates common deep features that are shared among multi-task modules. However, in a collaborative intelligence scenario, generating common deep features has two major issues. First, the deep features may inadvertently contain input information exposed to the downstream modules (violating input privacy). Second, the generated universal features expose a piece of collective information than what is intended for a certain task, in which features for one task can be utilized to perform another task (violating task privacy). This paper proposes a novel deep learning-based privacy-cognizant feature generation process called "MetaMorphosis" that limits inference capability to specific tasks at hand. To achieve this, we propose a channel squeeze-excitation based feature metamorphosis module, Cross-SEC, to achieve distinct attention of all tasks and a de-correlation loss function with differential-privacy to train a deep learning model that produces distinct privacy-aware features as an output for the respective tasks. With extensive experimentation on four datasets consisting of diverse images related to scene understanding and facial attributes, we show that MetaMorphosis outperforms recent adversarial learning and universal feature generation methods by guaranteeing privacy requirements in an efficient way for image and video analytics. CCS Concepts: • Computing methodologies → Computer vision tasks; Scene understanding; • Security and privacy → Privacy protections.

show abstract

Section: Motivation and Challengesmentioning

confidence: 99%

MetaMorphosis: Task-oriented Privacy Cognizant Feature Generation for Multi-task Learning

Arefeen

Uddin

et al. 2023

Proceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation

View full text Add to dashboard Cite

show abstract

“…Feature Pyramid Network [18] aggregates the multi-scale feature maps in a top-down fashion with progressive upsampling. Other network such as BiSeNet [19], ContextNet [20], GUN [21], and DSANet [22] utilized detail branch to capture low-level details in shallow layers.…”

Section: B Multi-scale and Context Aggregationmentioning

confidence: 99%

SPFNet:Subspace Pyramid Fusion Network for Semantic Segmentation

Elhassan¹,

Yang²,

Huang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The encoder-decoder structure has significantly improved performance in many vision tasks by fusing low-level and high-level feature maps. However, this approach can hardly extract sufficient context information for pixel-wise segmentation. In addition, extracting similar low-level features at multiple scales could lead to redundant information. To tackle these issues, we propose Subspace Pyramid Fusion Network (SPFNet). Specifically, we combine pyramidal module and context aggregation module to exploit the impact of multi-scale/global context information. At first, we construct a Subspace Pyramid Fusion Module (SPFM) based on Reduced Pyramid Pooling (RPP). Then, we propose the Efficient Global Context Aggregation (EGCA) module to capture discriminative features by fusing multi-level global context features. Finally, we add decoder-based subpixel convolution to retrieve the high-resolution feature maps, which can help select category localization details. SPFM learns separate RPP for each feature subspace to capture multi-scale feature representations, which is more useful for semantic segmentation. EGCA adopts shuffle attention mechanism to enhance communication across different sub-features. Experimental results on two well-known semantic segmentation datasets, including Camvid and Cityscapes, show that our proposed method is competitive with other state-of-the-art methods. The source code is available at 1

show abstract

“…Work [52] designs a lightweight dual attention structure that uses separable convolutions to simplify attention modeling between the spatial dimension and the channel dimension. The attention module introduced by DSANet [53] is also based on the dual attention structure, with a dilated expanded spatial attention module and a dilated channel attention module. In this paper, we propose an Swap Attention Module which is a spatial attention module that can effectively fuse the features of spatial detail branches and semantic branches.…”

Section: Related Workmentioning

confidence: 99%

SA‐BiSeNet: Swap attention bilateral segmentation network for real‐time inland waterways segmentation

Zhang

Bao

2022

IET Image Processing

View full text Add to dashboard Cite

The technology for autonomous navigation on inland waterways is worth investigating, and navigable water surface segmentation is a key part of this technology. Semantic segmentation methods based on deep learning are able to distinguish between water surface areas and non-water surface areas. However, existing semantic segmentation methods cannot meet the requirements of the water surface segmentation task in terms of both segmentation precision and real-time performance. In this study, a Swap Attention Bilateral Segmentation Network (SA-BiSeNet) is proposed to improve segmentation performance while ensuring model inference speed by better fusing the two features of the dual-branch down-sampling network using the attention mechanism. Specifically, an innovative Swap Attention Module is designed to model the dependency between the features of the spatial detail branch and the features of the semantic branches, thus expanding the receptive fields of the spatial detail and semantic branches to each other's global contexts. This design can effectively fuse features and thus enhance feature representation. Experiments were conducted on the inland waterway dataset USVInland to verify the performance of SA-BiSeNet in terms of segmentation precision and inference speed, and SA-BiSeNet achieved 93.65% Mean IoU and maintained the same level of fps as the baseline.

show abstract

DSANet: Dilated spatial attention for real-time semantic segmentation in urban street scenes

Cited by 79 publications

References 15 publications

MetaMorphosis: Task-oriented Privacy Cognizant Feature Generation for Multi-task Learning

MetaMorphosis: Task-oriented Privacy Cognizant Feature Generation for Multi-task Learning

SPFNet:Subspace Pyramid Fusion Network for Semantic Segmentation

SA‐BiSeNet: Swap attention bilateral segmentation network for real‐time inland waterways segmentation

Contact Info

Product

Resources

About