Self-Supervised Moving Vehicle Detection from Audio-Visual Cues

Zürn, Jannik; Burgard, Wolfram

doi:10.48550/arxiv.2201.12771

Cited by 2 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the recent advances in deep learning, perception systems of modern autonomous systems largely rely on convolutional neural networks (CNNs), in particular for the tasks of semantic segmentation [1] and object detection [2]. However, these two similar tasks are still often treated separately.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty-aware Panoptic Segmentation

Sirohi¹,

Marvi²,

Büscher³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Reliable scene understanding is indispensable for modern autonomous systems. Current learning-based methods typically try to maximize their performance based on segmentation metrics that only consider the quality of the segmentation. However, for the safe operation of a system in the real world it is crucial to consider the uncertainty in the prediction as well. In this work, we introduce the novel task of uncertainty-aware panoptic segmentation, which aims to predict per-pixel semantic and instance segmentations, together with per-pixel uncertainty estimates. We define two novel metrics to facilitate its quantitative analysis, the uncertaintyaware Panoptic Quality (uPQ) and the panoptic Expected Calibration Error (pECE). We further propose the novel topdown Evidential Panoptic Segmentation Network (EvPSNet) to solve this task. Our architecture employs a simple yet effective probabilistic fusion module that leverages the predicted uncertainties. Additionally, we propose a new Lovász evidential loss function to optimize the IoU for the segmentation utilizing the probabilities provided by deep evidential learning. Furthermore, we provide several strong baselines combining state-of-the-art panoptic segmentation networks with samplingfree uncertainty estimation techniques. Extensive evaluations show that our EvPSNet achieves the new state-of-the-art for the standard Panoptic Quality (PQ), as well as for our uncertaintyaware panoptic metrics.

show abstract

Section: Introductionmentioning

confidence: 99%

Uncertainty-aware Panoptic Segmentation

Sirohi¹,

Marvi²,

Büscher³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Street crossings are permitted to be crossed both by vehicles and by pedestrians. To allow for robust and safe navigation, autonomously operating robots in urban environments are required to localize nearby traffic participants accurately [1,2,3] and classify ground surfaces robustly. While autonomous vehicles typically require a binary distinction between road and non-road surfaces, mobile robots operating in pedestrian spaces must crucially be able to distinguish between sidewalks, roads, and road crossings in order to navigate urban environments safely [4,5,6].…”

Section: Introductionmentioning

confidence: 99%

TrackletMapper: Ground Surface Segmentation and Mapping from Traffic Participant Trajectories

Zürn¹,

Weber²,

Burgard³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Robustly classifying ground infrastructure such as roads and street crossings is an essential task for mobile robots operating alongside pedestrians. While many semantic segmentation datasets are available for autonomous vehicles, models trained on such datasets exhibit a large domain gap when deployed on robots operating in pedestrian spaces. Manually annotating images recorded from pedestrian viewpoints is both expensive and time-consuming. To overcome this challenge, we propose TrackletMapper, a framework for annotating ground surface types such as sidewalks, roads, and street crossings from object tracklets without requiring human-annotated data. To this end, we project the robot ego-trajectory and the paths of other traffic participants into the ego-view camera images, creating sparse semantic annotations for multiple types of ground surfaces from which a ground segmentation model can be trained. We further show that the model can be self-distilled for additional performance benefits by aggregating a ground surface map and projecting it into the camera images, creating a denser set of training annotations compared to the sparse tracklet annotations. We qualitatively and quantitatively attest our findings on a novel large-scale dataset for mobile robots operating in pedestrian areas. Code and dataset will be made available at http://trackletmapper.cs.uni-freiburg.de.

show abstract

Self-Supervised Moving Vehicle Detection from Audio-Visual Cues

Cited by 2 publications

References 21 publications

Uncertainty-aware Panoptic Segmentation

Uncertainty-aware Panoptic Segmentation

TrackletMapper: Ground Surface Segmentation and Mapping from Traffic Participant Trajectories

Contact Info

Product

Resources

About