KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Liao, Yiyi; Xie, Jun; Geiger, Andreas

doi:10.1109/tpami.2022.3179507

Cited by 297 publications

(91 citation statements)

References 101 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There are several datasets commonly used in training, validating, and evaluating vision-based autonomous vehicle-related technologies, such as vSLAM methods. These include KITTI [39], KITTI360 [40], The Oxford RobotCar Dataset [41], and recently the ApolloScape Dataset [42], Waymo Dataset [43]. One of the most used datasets, however, is still the KITTI [39] dataset that enables benchmarking individual solutions, too.…”

Section: Datasetsmentioning

confidence: 99%

ATDN vSLAM: An All-Through Deep Learning-Based Solution for Visual Simultaneous Localization and Mapping

Szántó

Bogár

Vajta

2022

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

In this paper, a novel solution is introduced for visual Simultaneous Localization and Mapping (vSLAM) that is built up of Deep Learning components. The proposed architecture is a highly modular framework in which each component offers state of the art results in their respective fields of vision-based Deep Learning solutions. The paper shows that with the synergic integration of these individual building blocks, a functioning and efficient all-through deep neural (ATDN) vSLAM system can be created. The Embedding Distance Loss function is introduced and using it the ATDN architecture is trained. The resulting system managed to achieve 4.4% translation and 0.0176 deg/m rotational error on a subset of the KITTI dataset. The proposed architecture can be used for efficient and low-latency autonomous driving (AD) aiding database creation as well as a basis for autonomous vehicle (AV) control.

show abstract

Section: Datasetsmentioning

confidence: 99%

ATDN vSLAM: An All-Through Deep Learning-Based Solution for Visual Simultaneous Localization and Mapping

Szántó

Bogár

Vajta

2022

Period. Polytech. Elec. Eng. Comp. Sci.

View full text Add to dashboard Cite

show abstract

“…Recently, the continuous emergence of panoramic semantic segmentation datasets [3], [19], [20], [58], [113] has facilitated the development of surrounding perception, and simulators have been used to generate multi-modal data [114], [115]. However, there is currently not a readily available large-scale semantic segmentation dataset for outdoor synthetic panoramas, considering that the OmniScape dataset [116] is still not released as of writing this paper.…”

Section: Synpass: Proposed Synthetic Datasetmentioning

confidence: 99%

“…USSS [129] relies on multi-source semi-supervised learning, while Seamless-Scene-Segmentation [130] uses instance-specific labels for auxiliary supervision. ISSAFE [131] merges training data from Cityscapes, KITTI-360 [113], and BDD [137] for robustifying segmentation. The semantic outputs of these models are projected to the 19 classes in DensePASS to be comparable with others.…”

Section: Ablation Of Unsupervised Domain Adaptationmentioning

confidence: 99%

Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation

Zhang¹,

Yang²,

Sheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we address panoramic semantic segmentation, which provides a full-view and dense-pixel understanding of surroundings in a holistic way. Panoramic segmentation is under-explored due to two critical challenges: (1) image distortions and object deformations on panoramas; (2) lack of annotations for training panoramic segmenters. To tackle these problems, we propose a Transformer for Panoramic Semantic Segmentation (Trans4PASS) architecture. First, to enhance distortion awareness, Trans4PASS, equipped with Deformable Patch Embedding (DPE) and Deformable MLP (DMLP) modules, is capable of handling object deformations and image distortions whenever (before or after adaptation) and wherever (shallow or deep levels) by design. We further introduce the upgraded Trans4PASS+ model, featuring DMLPv2 with parallel token mixing to improve the flexibility and generalizability in modeling discriminative cues. Second, we propose a Mutual Prototypical Adaptation (MPA) strategy for unsupervised domain adaptation. Third, aside from Pinhole-to-Panoramic (PIN2PAN) adaptation, we create a new dataset (SynPASS) with 9,080 panoramic images to explore a Synthetic-to-Real (SYN2REAL) adaptation scheme in 360 • imagery. Extensive experiments are conducted, which cover indoor and outdoor scenarios, and each of them is investigated with PIN2PAN and SYN2REAL regimens. Trans4PASS+ achieves state-of-the-art performances on four domain adaptive panoramic semantic segmentation benchmarks. Code is available at https://github.com/jamycheung/Trans4PASS.

show abstract

“…Modern scene segmenters are mostly designed to work with pinhole images on mainstream datasets such as Cityscapes [13] and Mapillary Vistas [57]. To enlarge the Field of View (FoV), many surrounding understanding platforms are based on fisheye images or multiple cameras [58], [59], [60], [61], [62]. However, this either comes with severe distortions in particular around the fisheye image borders or leads to being cumbersome with a lot of multi-camera calibration work.…”

Section: B Panoramic Scene Segmentationmentioning

confidence: 99%

Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning

Jaus

Yang

Stiefelhagen

2021

2021 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

In this work, we introduce panoramic panoptic segmentation as the most holistic scene understanding both in terms of field of view and image level understanding. A complete surrounding understanding provides a maximum of information to the agent, which is essential for any intelligent vehicle in order to make informed decisions in a safetycritical dynamic environment such as real-world traffic. In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers the learned features to a different domain. Using our proposed method, we manage to achieve significant improvements of over 5% measured in PQ over nonadapted models on our Wild Panoramic Panoptic Segmentation (WildPPS) dataset. We show that our proposed Panoramic Robust Feature (PRF) framework is not only suitable to improve performance on panoramic images but can be beneficial whenever model training and deployment are executed on data taken from different distributions. As an additional contribution, we publish WildPPS: The first panoramic panoptic image dataset to foster progress in surrounding perception.

show abstract

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Cited by 297 publications

References 101 publications

ATDN vSLAM: An All-Through Deep Learning-Based Solution for Visual Simultaneous Localization and Mapping

ATDN vSLAM: An All-Through Deep Learning-Based Solution for Visual Simultaneous Localization and Mapping

Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation

Panoramic Panoptic Segmentation: Towards Complete Surrounding Understanding via Unsupervised Contrastive Learning

Contact Info

Product

Resources

About