Importance-Aware Semantic Segmentation with Efficient Pyramidal Context Network for Navigational Assistant Systems

Xiang, Kaite; Wang, Kaiwei; Yang, Kailun

doi:10.1109/itsc.2019.8917096

Cited by 19 publications

(11 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traditional polarization-driven dense prediction frameworks were mainly dedicated to the detection of water hazards [39,40] or the perception in indoor scenes [41,42]. In our previous works, we investigated the impact of loss functions on water hazard segmentation [43], followed by a comparative study on high-recall semantic segmentation [44]. Inspired by [41], dense polarization maps are predicted from RGB images through deep learning [1].…”

Section: From Rgb-based To Multimodal Semantic Segmentationmentioning

confidence: 99%

Polarization-driven semantic segmentation via efficient attention-bridged fusion

2021

Self Cite

View full text Add to dashboard Cite

Semantic Segmentation (SS) is promising for outdoor scene perception in safetycritical applications like autonomous vehicles, assisted navigation and so on. However, traditional SS is primarily based on RGB images, which limits the reliability of SS in complex outdoor scenes, where RGB images lack necessary information dimensions to fully perceive unconstrained environments. As preliminary investigation, we examine SS in an unexpected obstacle detection scenario, which demonstrates the necessity of multimodal fusion. Thereby, in this work, we present EAFNet, an Efficient Attention-bridged Fusion Network to exploit complementary information coming from different optical sensors. Specifically, we incorporate polarization sensing to obtain supplementary information, considering its optical characteristics for robust representation of diverse materials. By using a single-shot polarization sensor, we build the first RGB-P dataset which consists of 394 annotated pixel-aligned RGB-Polarization images. A comprehensive variety of experiments shows the effectiveness of EAFNet to fuse polarization and RGB information, as well as the flexibility to be adapted to other sensor combination scenarios.

show abstract

Section: From Rgb-based To Multimodal Semantic Segmentationmentioning

confidence: 99%

Polarization-driven semantic segmentation via efficient attention-bridged fusion

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…For semantic segmentation, a method to achieve a higher recall rate is proposed in [15] based on the loss function, classifier and decision rule for a real-time neural network. The similar approach in [16] uses an importance-aware loss function to improve the networks' reliability. In [17], the differences between the Maximum Likelihood and the Bayes decision rule are considered to reduce false negatives of minority classes by introducing class priors which assign larger weight to underrepresented classes.…”

Section: Related Workmentioning

confidence: 99%

False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates

Maag¹

2021

Preprint

View full text Add to dashboard Cite

Instance segmentation of images is an important tool for automated scene understanding. Neural networks are usually trained to optimize their overall performance in terms of accuracy. Meanwhile, in applications such as automated driving, an overlooked pedestrian seems more harmful than a falsely detected one. In this work, we present a false negative detection method for image sequences based on inconsistencies in time series of tracked instances given the availability of image sequences in online applications. As the number of instances can be greatly increased by this algorithm, we apply a false positive pruning using uncertainty estimates aggregated over instances. To this end, instance-wise metrics are constructed which characterize uncertainty and geometry of a given instance or are predicated on depth estimation. The proposed method serves as a post-processing step applicable to any neural network that can also be trained on single frames only. In our tests, we obtain an improved trade-off between false negative and false positive instances by our fused detection approach in comparison to the use of an ordinary score value provided by the instance segmentation network during inference.

show abstract

“…Semantic segmentation is important in understanding the content of images and finding target objects, and this technique is vital for the field of automatic driving. 20,21 Currently, most of state-of-the-art semantic segmentation models are based on fully convolutional end-to-end networks. 22 Inspired by SegNet, 23 semantic segmentation models usually follow an encoder-decoder network architecture.…”

Section: Semantic Understanding Of the Road Scenementioning

confidence: 99%

See clearer at night: towards robust nighttime semantic segmentation through day-night image conversion

Sun

Wang

Yang

et al. 2019

Artificial Intelligence and Machine Learning in Defense Applications

Self Cite

View full text Add to dashboard Cite

In recent years, intelligent driving navigation and security monitoring have made considerable progress with the help of deep Convolutional Neural Networks (CNNs). As one of the state-of-the-art perception approaches, semantic segmentation unifies distinct detection tasks widely desired by both autonomous driving and security monitoring. Currently, semantic segmentation shows remarkable efficiency and reliability in standard scenarios such as daytime scenes with favorable illumination conditions. However, in face of adverse conditions such as the nighttime, semantic segmentation loses its accuracy significantly. One of the main causes of the problem is the lack of sufficient annotated segmentation datasets of nighttime scenes. In this paper, we propose a framework to alleviate the accuracy decline when semantic segmentation is taken to adverse conditions by using Generative Adversarial Networks (GANs). To bridge the daytime and nighttime image domains, we made key observation that compared to datasets in adverse conditions, there are considerable amount of segmentation datasets in standard conditions such as BDD and our collected ZJU datasets. Our GAN-based nighttime semantic segmentation framework includes two methods. In the first method, GANs were used to translate nighttime images to the daytime, thus semantic segmentation can be performed using robust models already trained on daytime datasets. In another method, we use GANs to translate different ratio of daytime images in the dataset to the nighttime but still with their labels. In this sense, synthetic nighttime segmentation datasets can be generated to yield models prepared to operate at nighttime conditions robustly. In our experiment, the later method significantly boosts the performance at the nighttime evidenced by quantitative results using Intersection over Union (IoU) and Pixel Accuracy (Acc). We show that the performance varies with respect to the proportion of synthetic nighttime images in the dataset, where the sweet spot corresponds to most robust performance across the day and night. The proposed framework not only makes contribution to the optimization of visual perception in intelligent vehicles, but also can be applied to diverse navigational assistance systems.

show abstract

Importance-Aware Semantic Segmentation with Efficient Pyramidal Context Network for Navigational Assistant Systems

Cited by 19 publications

References 25 publications

Polarization-driven semantic segmentation via efficient attention-bridged fusion

Polarization-driven semantic segmentation via efficient attention-bridged fusion

False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates

See clearer at night: towards robust nighttime semantic segmentation through day-night image conversion

Contact Info

Product

Resources

About