Joint Semantic Segmentation and Boundary Detection Using Iterative Pyramid Contexts

Zhen, Mingmin; Wang, Jinglu; Zhou, Lei; Li, Shiwei; Shen, Tianwei; Shang, Jiaxiang; Fang, Tian; Quan, Long

doi:10.1109/cvpr42600.2020.01368

Cited by 119 publications

(71 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, several methods are starting to explicitly model boundary detection as an independent subtask in parallel with semantic segmentation for sharper results. Takikawa et al [17] and Zhen et al [31] specially design a boundary stream and couple the two tasks of boundary and semantics modeling. Li et al [12] point out that the object boundary and body parts correspond to the high frequency and low frequency information of an image, respectively, based on which they decouple the body and edge with diverse supervisions.…”

Section: Boundary In Semantic Segmentationmentioning

confidence: 99%

Boundary Guided Context Aggregation for Semantic Segmentation

Ma¹,

Yang²,

Huang³

2021

Preprint

View full text Add to dashboard Cite

The recent studies on semantic segmentation are starting to notice the significance of the boundary information, where most approaches see boundaries as the supplement of semantic details. However, simply combing boundaries and the mainstream features cannot ensure a holistic improvement of semantics modeling. In contrast to the previous studies, we exploit boundary as a significant guidance for context aggregation to promote the overall semantic understanding of an image. To this end, we propose a Boundary guided Context Aggregation Network (BCANet), where a Multi-Scale Boundary extractor (MSB) borrowing the backbone features at multiple scales is specifically designed for accurate boundary detection. Based on which, a Boundary guided Context Aggregation module (BCA) improved from Non-local network is further proposed to capture long-range dependencies between the pixels in the boundary regions and the ones inside the objects. By aggregating the context information along the boundaries, the inner pixels of the same category achieve mutual gains and therefore the intra-class consistency is enhanced. We conduct extensive experiments on the Cityscapes and ADE20K databases, and comparable results are achieved with the state-of-the-art methods, clearly demonstrating the effectiveness of the proposed one. Our code is available at https:// github.com/mahaoxiang822/Boundary-Guided-Context-Aggregation.

show abstract

Section: Boundary In Semantic Segmentationmentioning

confidence: 99%

Boundary Guided Context Aggregation for Semantic Segmentation

Ma¹,

Yang²,

Huang³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…At present, of the multimodel method can be used for semantic segmentation and object detection [37], but the multimodel method leads to too much training cost.…”

Section: Attention-wise Modulementioning

confidence: 99%

“…Instead of calculating attention scores directly, we decompose the process into a learning channel and position attention information. The individual attention score generation process of the feature map is less than that of [37]; thus, it can be regarded as a plug-and-play module for the existing basic convolutional neural networks. Hu et al [41] introduced a compact module to take advantage of the relationship between channels.…”

Section: Edge Prior and Attention Mechanismmentioning

confidence: 99%

EAWNet: An Edge Attention‐Wise Objector for Real‐Time Visual Internet of Things

Zhang

Chen

Yin

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

With the upgrading of the high-performance image processing platform and visual internet of things sensors, VIOT is widely used in intelligent transportation, autopilot, military reconnaissance, public safety, and other fields. However, the outdoor visual internet of things system is very sensitive to the weather and unbalanced scale of latent object. The performance of supervised learning is often limited by the disturbance of abnormal data. It is difficult to collect all classes from limited historical instances. Therefore, in terms of the anomaly detection images, fast and accurate artificial intelligence-based object detection technology has become a research hot spot in the field of intelligent vision internet of things. To this end, we propose an efficient and accurate deep learning framework for real-time and dense object detection in VIOT named the Edge Attention-wise Convolutional Neural Network (EAWNet) with three main features. First, it can identify remote aerial and daily scenery objects fast and accurately in terms of an unbalanced category. Second, edge prior and rotated anchor are adopted to enhance the efficiency of detection in edge computing internet. Third, our EAWNet network uses an edge sensing object structure, makes full use of an attention mechanism to dynamically screen different kinds of objects, and performs target recognition on multiple scales. The edge recovery effect and target detection performance for long-distance aerial objects were significantly improved. We explore the efficiency of various architectures and fine tune the training process using various backbone and data enhancement strategies to increase the variety of the training data and overcome the size limitation of input images. Extensive experiments and comprehensive evaluation on COCO and large-scale DOTA datasets proved the effectiveness of this framework that achieved the most advanced performance in real-time VIOT object detection.

show abstract

“…Acuna et al propose a new approach to learn sharper and more accurate semantic boundaries [40]. By treating boundary detection as the dual task of semantic segmentation, a new loss function with a boundary consistency constraint to improve the boundary pixel accuracy for semantic segmentation is designed [41]. The work in [42] proposes a content-adaptive downsampling technique that learns to favor sampling locations near semantic boundaries of target classes.…”

Section: Related Workmentioning

confidence: 99%

Polyline Generative Navigable Space Segmentation for Autonomous Visual Navigation

Zheng¹,

Ding²,

Crandall³

et al. 2021

Preprint

View full text Add to dashboard Cite

Detecting navigable space is a fundamental capability for mobile robots navigating in unknown or unmapped environments. In this work, we treat the visual navigable space segmentation as a scene decomposition problem and propose Polyline Segmentation Variational AutoEncoder Networks (PSV-Nets), a representation learning based framework to enable robots to learn the navigable space segmentation in an unsupervised manner. Current segmentation techniques heavily rely on supervised learning strategies which demand a large amount of pixel-level annotated images. In contrast, the proposed framework leverages a generative model -Variational AutoEncoder (VAE) and an AutoEncoder (AE) to learn a polyline representation that compactly outlines the desired navigable space boundary in an unsupervised way. We also propose a visual receding horizon planning method that uses the learned navigable space and a Scaled Euclidean Distance Field (SEDF) to achieve autonomous navigation without an explicit map. Through extensive experiments, we have validated that the proposed PSV-Nets can learn the visual navigable space with high accuracy, even without any single label. We also show that the prediction of the PSV-Nets can be further improved with a small number of labels (if available) and can significantly outperform the state-of-the-art fully supervised learning based segmentation methods.

show abstract

Joint Semantic Segmentation and Boundary Detection Using Iterative Pyramid Contexts

Cited by 119 publications

References 24 publications

Boundary Guided Context Aggregation for Semantic Segmentation

Boundary Guided Context Aggregation for Semantic Segmentation

EAWNet: An Edge Attention‐Wise Objector for Real‐Time Visual Internet of Things

Polyline Generative Navigable Space Segmentation for Autonomous Visual Navigation

Contact Info

Product

Resources

About