Semantic Concept Testing in Autonomous Driving by Extraction of Object-Level Annotations from CARLA

Gannamaneni, Sujan Sai; Houben, Sebastian; Akila, Maram

doi:10.1109/iccvw54120.2021.00117

Cited by 12 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These datasets are valuable for benchmarking cross-domain adaptation methods but do not allow for additional creation of corner-case data. Here, autonomous driving simulators such as Carla [DRC+17] and the LGSVL simulator [RST+20] prove to be beneficial [LGH+21b;GHA21]. Procedural methods for road generation [PJX+20] can enhance the capabilities of these methods.…”

Section: Visual Perception Datasetsmentioning

confidence: 99%

Training and Validation of Visual Perception Functions for Autonomous Driving with Synthetic Data

Hagn

View full text Add to dashboard Cite

This work deals with the usability of synthetically generated data for training and validation of visual perception functions applied in autonomous driving. Synthetically generated images allow the creation of safety critical scenarios which are potentially dangerous to capture in the real-world and additionally deliver pixel perfect ground truth annotations. However, applying synthetic images to perception functions trained on real-world data poses the problem of bridging the domain gap. This is true for both training and validating with synthetic images. Therefore, the domain gap has to be sufficiently understood to generate synthetically images viable to be used for training and validation. A new domain discrepancy metric is introduced and applied to optimize the parameters of a realistic sensor simulation effectively reducing the domain gap. Several influence factors on the domain gap are disentangled. The visual detection impairing factors are introduced and shown to have a high influence on the detectability of pedestrians. Additionally, these factors are used to calibrate a weighting loss function to increase the perception performance on real-world pedestrians. New methods for perception validation are introduced. The deep variational data synthesis and the classification of visual detection impairing factors. While the former method searches for perception faults by parameterized probabilistic image creation, the latter method detects perception faults by the disagreement of a detectability classifier and the actual detection result. The findings of both training and validating were influencing the creation of two synthetic validation datasets, VALERIE and SynPeDS.

show abstract

Section: Visual Perception Datasetsmentioning

confidence: 99%

Training and Validation of Visual Perception Functions for Autonomous Driving with Synthetic Data

Hagn

View full text Add to dashboard Cite

show abstract

Validation of Pedestrian Detectors by Classification of Visual Detection Impairing Factors

Hagn¹,

Grau²

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Assessing systematic weaknesses of DNNs using counterfactuals

Gannamaneni,

Mock,

Akila

2024

AI Ethics

View full text Add to dashboard Cite

With the advancement of DNNs into safety-critical applications, testing approaches for such models have gained more attention. A current direction is the search for and identification of systematic weaknesses that put safety assumptions based on average performance values at risk. Such weaknesses can take on the form of (semantically coherent) subsets or areas in the input space where a DNN performs systematically worse than its expected average. However, it is non-trivial to attribute the reason for such observed low performances to the specific semantic features that describe the subset. For instance, inhomogeneities within the data w.r.t. other (non-considered) attributes might distort results. However, taking into account all (available) attributes and their interaction is often computationally highly expensive. Inspired by counterfactual explanations, we propose an effective and computationally cheap algorithm to validate the semantic attribution of existing subsets, i.e., to check whether the identified attribute is likely to have caused the degraded performance. We demonstrate this approach on an example from the autonomous driving domain using highly annotated simulated data, where we show for a semantic segmentation model that (i) performance differences among the different pedestrian assets exist, but (ii) only in some cases is the asset type itself the reason for this reduction in the performance.

show abstract

Semantic Concept Testing in Autonomous Driving by Extraction of Object-Level Annotations from CARLA

Cited by 12 publications

References 16 publications

Training and Validation of Visual Perception Functions for Autonomous Driving with Synthetic Data

Training and Validation of Visual Perception Functions for Autonomous Driving with Synthetic Data

Validation of Pedestrian Detectors by Classification of Visual Detection Impairing Factors

Assessing systematic weaknesses of DNNs using counterfactuals

Contact Info

Product

Resources

About