SynWoodScape: Synthetic Surround-View Fisheye Camera Dataset for Autonomous Driving

Sekkat, Ahmed Rida; Dupuis, Yohan; Kumar, Varun; Rashed, Hazem; Yogamani, Senthil; Vasseur, Pascal; Honeiné, Paul

doi:10.1109/lra.2022.3188106

Cited by 37 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The polynomial Brown-Conrady model (including calibration methods), is implemented in many commonly used software packages and libraries (e.g., MATLAB [88], [89]). The popular Woodscape datasets [39], [90], examined closely in this paper, use a full fourth-order polynomial model for fisheye projection:…”

Section: E Geometric Distortionmentioning

confidence: 99%

“…This can also be possible where there is a lack of ground-truth data in real-world images for training computer vision. For example, as of the writing of this survey, SynWoodscape [90] is the only publicly available synthetic fisheye dataset with pixel-wise optical flow ground truth. For the original Woodscape, it was not possible to obtain optical flow ground truth in the real scenes, and synthetic data was identified as the only feasible solution [39].…”

Section: A the Argument For Simulationsmentioning

confidence: 99%

“…To overcome this, to the best of the authors' knowledge, all fisheye models created in CARLA use an intermediate cube map for the creation of the fisheye image [90], [115], [116], [117], shown in Figure 17. The cube map can then be mapped to a fisheye image using any fisheye model.…”

Section: B Model-driven Simulatorsmentioning

confidence: 99%

See 2 more Smart Citations

Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges

Jakab,

Deegan,

Sharma

et al. 2024

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

In this paper, we provide a survey on automotive surround-view fisheye optics, with an emphasis on the impact of optical artifacts on computer vision tasks in autonomous driving and ADAS. The automotive industry has advanced in applying state-of-the-art computer vision to enhance road safety and provide automated driving functionality. When using camera systems on vehicles, there is a particular need for a wide field of view to capture the entire vehicle's surroundings, in areas such as low-speed maneuvering, automated parking, and cocoon sensing. However, one crucial challenge in surround-view cameras is the strong optical aberrations of the fisheye camera, which is an area that has received little attention in the literature. Additionally, a comprehensive dataset is needed for testing safety-critical scenarios in vehicle automation. The industry has turned to simulation as a cost-effective strategy for creating synthetic datasets with surround-view camera imagery. We examine different simulation methods (such as model-driven and data-driven simulations) and discuss the simulators' ability (or lack thereof) to model real-world optical performance. Overall, this paper highlights the optical aberrations in automotive fisheye datasets, and the limitations of optical reality in simulated fisheye datasets, with a focus on computer vision in surround-view optical systems.

show abstract

Section: E Geometric Distortionmentioning

confidence: 99%

Section: A the Argument For Simulationsmentioning

confidence: 99%

See 1 more Smart Citation

Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges

Jakab,

Deegan,

Sharma

et al. 2024

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

show abstract

“…Our experiments are built on the WoodScape dataset [25], a widely used and challenging fisheye image dataset that can handle a variety of tasks. The semantic segmentation part of it contains 10,000 images covering 10 categories.…”

Section: Dataset and Implementationsmentioning

confidence: 99%

DSA: Deformable Segmentation Attention for Multi-Scale Fisheye Image Segmentation

Jiang,

Xu,

Liu

et al. 2023

Electronics

View full text Add to dashboard Cite

With a larger field of view (FOV) than ordinary images, fisheye images are becoming mainstream in the field of autonomous driving. However, the severe distortion problem of fisheye images also limits its application. The performance of neural networks designed for narrow FOV images degrades drastically for fisheye images, and the use of large composite models can improve the performance, but it brings huge time overhead and hardware costs. Therefore, we decided to balance real time and accuracy by designing the deformable segmentation attention(DSA) module, a generalpurpose architecture based on a deformable attention mechanism and a spatial pyramid architecture. The deformable mechanism serves to accurately extract feature information from fisheye images, together with attention to learn the global context and the spatial pyramid structure to balance multiscale feature information, thus improving the perception of fisheye images by traditional networks without increasing the amount of excessive computation. Lightweight networks such as SegNeXt equipped with the DSA module enable effective and rapid multi-scale segmentation of fisheye images in complex scenes. Our architecture achieves outstanding results on the WoodScape dataset, while our ablation experiments demonstrate the effectiveness of various parts of the architecture.

show abstract

“…Recently, the continuous emergence of panoramic semantic segmentation datasets [3], [19], [20], [58], [113] has facilitated the development of surrounding perception, and simulators have been used to generate multi-modal data [114], [115]. However, there is currently not a readily available large-scale semantic segmentation dataset for outdoor synthetic panoramas, considering that the OmniScape dataset [116] is still not released as of writing this paper.…”

Section: Synpass: Proposed Synthetic Datasetmentioning

confidence: 99%

Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation

Zhang¹,

Yang²,

Sheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we address panoramic semantic segmentation, which provides a full-view and dense-pixel understanding of surroundings in a holistic way. Panoramic segmentation is under-explored due to two critical challenges: (1) image distortions and object deformations on panoramas; (2) lack of annotations for training panoramic segmenters. To tackle these problems, we propose a Transformer for Panoramic Semantic Segmentation (Trans4PASS) architecture. First, to enhance distortion awareness, Trans4PASS, equipped with Deformable Patch Embedding (DPE) and Deformable MLP (DMLP) modules, is capable of handling object deformations and image distortions whenever (before or after adaptation) and wherever (shallow or deep levels) by design. We further introduce the upgraded Trans4PASS+ model, featuring DMLPv2 with parallel token mixing to improve the flexibility and generalizability in modeling discriminative cues. Second, we propose a Mutual Prototypical Adaptation (MPA) strategy for unsupervised domain adaptation. Third, aside from Pinhole-to-Panoramic (PIN2PAN) adaptation, we create a new dataset (SynPASS) with 9,080 panoramic images to explore a Synthetic-to-Real (SYN2REAL) adaptation scheme in 360 • imagery. Extensive experiments are conducted, which cover indoor and outdoor scenarios, and each of them is investigated with PIN2PAN and SYN2REAL regimens. Trans4PASS+ achieves state-of-the-art performances on four domain adaptive panoramic semantic segmentation benchmarks. Code is available at https://github.com/jamycheung/Trans4PASS.

show abstract

SynWoodScape: Synthetic Surround-View Fisheye Camera Dataset for Autonomous Driving

Cited by 37 publications

References 25 publications

Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges

Surround-View Fisheye Optics in Computer Vision and Simulation: Survey and Challenges

DSA: Deformable Segmentation Attention for Multi-Scale Fisheye Image Segmentation

Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation

Contact Info

Product

Resources

About