Out-of-Distribution Detection for Automotive Perception

Nitsch, Julia; Itkina, Masha; Senanayake, Ransalu; Nieto, Juan; Schmidt, Max; Siegwart, Roland; Kochenderfer, Mykel J.; Cadena, César

doi:10.1109/itsc48978.2021.9564545

Cited by 38 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Perception data corruption results from environmental disturbance/interference is inevitable in self-driving applications. In [10], a Generative Adversarial Network (GAN) is presented to detect out-of-distribution data. Degradation of visual data [16] and LiDAR point cloud [17] in various weather conditions also have been studied.…”

Section: B Data Corruption In Self-drivingmentioning

confidence: 99%

“…For example, challenging weather conditions (such as fog and heavy rain), various illumination conditions and several corner cases (such as light reflection and vehicle-like figures in the traffic scenario) may decrease the detection and tracking performance, thus leading to mis-perception. 2) Current methods highly rely on deep-learning models, which can fail on the input data not well represented by the training dataset [10]. It happens sometimes that a non-existent vehicle is mis-detected or an existing vehicle is tracked to a wrong position.…”

Section: Introductionmentioning

confidence: 99%

“…Resolving the ghost issue is very challenging due to the complex external disturbances and various attacking methods. Existing robust methods are usually only effective on one specific interference and cannot resist unseen disturbances [10]. In this paper, we do not explicitly detect the interference from the aspect of perception, instead, we aim to represent the reliability by using the relation learning and let the navigation model to learn how to aggregate the information with uncertainties by itself.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigation

Jin¹,

Han²

2022

Preprint

View full text Add to dashboard Cite

Environmental disturbances, such as sensor data noises, various lighting conditions, challenging weathers and external adversarial perturbations, are inevitable in real selfdriving applications. Existing researches and testings have shown that they can severely influence the vehicle's perception ability and performance, one of the main issue is the false positive detection, i.e., the "ghost" object which is not real existed or occurs in the wrong position (such as a non-existent vehicle). Traditional navigation methods tend to avoid every detected objects for safety, however, avoiding a "ghost" object may lead the vehicle into a even more dangerous situation, such as a sudden break on the highway. Considering the various disturbance types, it is difficult to address this issue at the perceptual aspect. A potential solution is to detect the ghost through relation learning among the whole scenario and develop an integrated end-to-end navigation system. Our underlying logic is that the behavior of all vehicles in the scene is influenced by their neighbors, and normal vehicles behave in a logical way, while "ghost" vehicles do not. By learning the spatiotemporal relation among surrounding vehicles, an information reliability representation is learned for each detected vehicle and then a robot navigation network is developed. In contrast to existing works, we encourage the network to learn how to represent the reliability and how to aggregate all the information with uncertainties by itself, thus increasing the efficiency and generalizability. To the best of the authors' knowledge, this paper provides the first work on using graph relation learning to achieve end-to-end robust navigation in the presence of ghost vehicles. Simulation results in the CARLA platform demonstrate the feasibility and effectiveness of the proposed method in various scenarios.

show abstract

Section: B Data Corruption In Self-drivingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigation

Jin¹,

Han²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For more on these methods, refer to the survey by Akhtar et al [17]. Generative adversarial networks (GANs) have also been used to generate attacks on object detection and to identify out-of-distribution examples in object detection [18], [19]. These methods require ample data to generate realistic attacks which may not be available and do not consider temporal sequences of observations.…”

Section: B Adversarial Attacksmentioning

confidence: 99%

How Do We Fail? Stress Testing Perception in Autonomous Vehicles

Delecki¹,

Itkina²,

Lange³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Autonomous vehicles (AVs) rely on environment perception and behavior prediction to reason about agents in their surroundings. These perception systems must be robust to adverse weather such as rain, fog, and snow. However, validation of these systems is challenging due to their complexity and dependence on observation histories. This paper presents a method for characterizing failures of LiDAR-based perception systems for AVs in adverse weather conditions. We develop a methodology based in reinforcement learning to find likely failures in object tracking and trajectory prediction due to sequences of disturbances. We apply disturbances using a physicsbased data augmentation technique for simulating LiDAR point clouds in adverse weather conditions. Experiments performed across a wide range of driving scenarios from a real-world driving dataset show that our proposed approach finds high likelihood failures with smaller input disturbances compared to baselines while remaining computationally tractable. Identified failures can inform future development of robust perception systems for AVs.

show abstract

“…This is also referred to as the open-set setting [9]. A variety of different approaches for that problem exist [6], [9], [11], [12]. While AVs consist of a set of different sensors, such as lidars, radars, and cameras, most of the existing work on anomaly detection focuses on one sensor modality [13].…”

Section: Introductionmentioning

confidence: 99%

Multimodal Detection of Unknown Objects on Roads for Autonomous Driving

Bogdoll¹,

Enrico²,

Nitsche³

et al. 2022

Preprint

View full text Add to dashboard Cite

Tremendous progress in deep learning over the last years has led towards a future with autonomous vehicles on our roads. Nevertheless, the performance of their perception systems is strongly dependent on the quality of the utilized training data. As these usually only cover a fraction of all object classes an autonomous driving system will face, such systems struggle with handling the unexpected. In order to safely operate on public roads, the identification of objects from unknown classes remains a crucial task. In this paper, we propose a novel pipeline to detect unknown objects. Instead of focusing on a single sensor modality, we make use of lidar and camera data by combining state-of-the art detection models in a sequential manner. We evaluate our approach on the Waymo Open Perception Dataset and point out current research gaps in anomaly detection.

show abstract

Out-of-Distribution Detection for Automotive Perception

Cited by 38 publications

References 19 publications

Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigation

Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigation

How Do We Fail? Stress Testing Perception in Autonomous Vehicles

Multimodal Detection of Unknown Objects on Roads for Autonomous Driving

Contact Info

Product

Resources

About