How Do We Fail? Stress Testing Perception in Autonomous Vehicles

Abstract

Autonomous vehicles (AVs) rely on environment perception and behavior prediction to reason about agents in their surroundings. These perception systems must be robust to adverse weather such as rain, fog, and snow. However, validation of these systems is challenging due to their complexity and dependence on observation histories. This paper presents a method for characterizing failures of LiDAR-based perception systems for AVs in adverse weather conditions. We develop a methodology based in reinforcement learning to find likely failures in object tracking and trajectory prediction due to sequences of disturbances. We apply disturbances using a physicsbased data augmentation technique for simulating LiDAR point clouds in adverse weather conditions. Experiments performed across a wide range of driving scenarios from a real-world driving dataset show that our proposed approach finds high likelihood failures with smaller input disturbances compared to baselines while remaining computationally tractable. Identified failures can inform future development of robust perception systems for AVs.

Monitoring of Perception Systems: Deterministic, Probabilistic, and Learning-based Fault Detection and Identification

“…Current approaches [41,42,43] consider high-level abstractions of perception [17,44,45] or rely on simulation to assert the true state of the world [41,42,46]. Other approaches focus on adversarial attacks for neural-network-based object detection [47,48,49]; these methods derive bounds on the magnitude of the perturbation that induces incorrect detection result, and are typically used off-line [50].…”