A Comparative Study of Real-Time Semantic Segmentation for Autonomous Driving

Siam, Mennatullah; Gamal, Mostafa; Abdel-Razek, Moemen; Yogamani, Senthil; Jägersand, Martin; Zhang, Hong

doi:10.1109/cvprw.2018.00101

Cited by 144 publications

(82 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Semantic segmentation is a long-standing challenging task in computer vision, aiming to predict pixel-wise semantic labels in an image accurately. This task is exceptionally important to tons of real-world applications, such as autonomous driving [27,28], medical diagnosing [52,53], etc. In recent years, the developments of deep neural networks encourage the emergence of a series of works [1,5,18,26,40,43,47].…”

Section: Introductionmentioning

confidence: 99%

Asymmetric Non-Local Neural Networks for Semantic Segmentation

Zhu

Mengdu

Bai

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

672

362

View full text Add to dashboard Cite

The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation. In this paper, we present Asymmetric Non-local Neural Network to semantic segmentation, which has two prominent components: Asymmetric Pyramid Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB). APNB leverages a pyramid sampling module into the nonlocal block to largely reduce the computation and memory consumption without sacrificing the performance. AFNB is adapted from APNB to fuse the features of different levels under a sufficient consideration of long range dependencies and thus considerably improves the performance. Extensive experiments on semantic segmentation benchmarks demonstrate the effectiveness and efficiency of our work. In particular, we report the state-of-the-art performance of 81.3 mIoU on the Cityscapes test set. For a 256 × 128 input, APNB is around 6 times faster than a non-local block on GPU while 28 times smaller in GPU running memory occupation. Code is available at: https://github.com/ MendelXu/ANN.git. * Equal contribution † Corresponding author < l a t e x i t s h a 1 _ b a s e 6 4 = " h Z k z g Y O / g V + m e P P y R w 8 d X I Q M H C U = " > A A A B 6 3 i c b Z D L S g M x F I Z P 6 q 3 W W 9 W l m 2 A R X E i Z c a P L g h u X F e w F 2 q F k 0 k w n N M k M S U Y o Q 1 / B j Q t F 3 P p C 7 n w b M + 0 s t P W H

show abstract

Section: Introductionmentioning

confidence: 99%

Asymmetric Non-Local Neural Networks for Semantic Segmentation

Zhu

Mengdu

Bai

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

672

362

View full text Add to dashboard Cite

show abstract

“…In autonomous agents in general, the use of semantic segmentation has been studied fairly well in autonomous road vehicles. Siam et al [183] have done an in-depth comparison of such semantic segmentation methods for autonomous driving and proposed a real-time segmentation benchmarking framework.…”

Section: A Semantic Segmentationmentioning

confidence: 99%

Collaborative Multi-Robot Search and Rescue: Planning, Coordination, Perception, and Active Vision

et al. 2020

View full text Add to dashboard Cite

Search and rescue (SAR) operations can take significant advantage from supporting autonomous or teleoperated robots and multi-robot systems. These can aid in mapping and situational assessment, monitoring and surveillance, establishing communication networks, or searching for victims. This paper provides a review of multi-robot systems supporting SAR operations, with system-level considerations and focusing on the algorithmic perspectives for multi-robot coordination and perception. This is, to the best of our knowledge, the first survey paper to cover (i) heterogeneous SAR robots in different environments, (ii) active perception in multi-robot systems, while (iii) giving two complementary points of view from the multi-agent perception and control perspectives. We also discuss the most significant open research questions: shared autonomy, sim-to-real transferability of existing methods, awareness of victims' conditions, coordination and interoperability in heterogeneous multi-robot systems, and active perception. The different topics in the survey are put in the context of the different challenges and constraints that various types of robots (ground, aerial, surface, or underwater) encounter in different SAR environments (maritime, urban, wilderness, or other post-disaster scenarios). The objective of this survey is to serve as an entry point to the various aspects of multi-robot SAR systems to researchers in both the machine learning and control fields by giving a global overview of the main approaches being taken in the SAR robotics area.

show abstract

“…Due to run-time complexity semantic segmentation typically takes hundreds of milliseconds to run with only more recent models edging closer to real time implementations [4], [5]. However, this problem is only exacerbated in application scenarios especially on deployable or embedded systems [6]. Considering real-time semantic segmentation has important applications, e.g., street scene understanding, autonomous driving and augmented reality for wearables.…”

Section: Introductionmentioning

confidence: 99%

SpikeSEG: Spiking Segmentation via STDP Saliency Mapping

Kirkland

Caterina

Soraghan

et al. 2020

2020 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Taking inspiration from the structure and behaviour of the human visual system and using the Transposed Convolution and Saliency Mapping methods of Convolutional Neural Networks (CNN), a spiking event-based image segmentation algorithm, SpikeSEG is proposed. The approach makes use of both spike-based imaging and spike-based processing, where the images are either standard images converted to spiking images or they are generated directly from a neuromorphic event driven sensor, and then processed using a spiking fully convolutional neural network. The spiking segmentation method uses the spike activations through time within the network to trace back any outputs from saliency maps, to the exact pixel location. This not only gives exact pixel locations for spiking segmentation, but with low latency and computational overhead. SpikeSEG is the first spiking event-based segmentation network and over three experiment test achieves promising results with 96% accuracy overall and a 74% mean intersection over union for the segmentation, all within an event by event-based framework.

show abstract

A Comparative Study of Real-Time Semantic Segmentation for Autonomous Driving

Cited by 144 publications

References 26 publications

Asymmetric Non-Local Neural Networks for Semantic Segmentation

Asymmetric Non-Local Neural Networks for Semantic Segmentation

Collaborative Multi-Robot Search and Rescue: Planning, Coordination, Perception, and Active Vision

SpikeSEG: Spiking Segmentation via STDP Saliency Mapping

Contact Info

Product

Resources

About