Monitoring of Perception Systems: Deterministic, Probabilistic, and Learning-based Fault Detection and Identification

Abstract

This paper investigates runtime monitoring of perception systems. Perception is a critical component of high-integrity applications of robotics and autonomous systems, such as self-driving cars. In these applications, failure of perception systems may put human life at risk, and a broad adoption of these technologies requires the development of methodologies to guarantee and monitor safe operation. Despite the paramount importance of perception, currently there is no formal approach for system-level perception monitoring. In this paper, we formalize the problem of runtime fault detection and identification in perception systems and present a framework to model diagnostic information using a diagnostic graph. We then provide a set of deterministic, probabilistic, and learning-based algorithms that use diagnostic graphs to perform fault detection and identification. Moreover, we investigate fundamental limits and provide deterministic and probabilistic guarantees on the fault detection and identification results. We conclude the paper with an extensive experimental evaluation, which recreates several realistic failure modes in the LGSVL opensource autonomous driving simulator, and applies the proposed system monitors to a state-of-the-art autonomous driving software stack (Baidu's Apollo Auto).

Introspection of DNN-Based Perception Functions in Automated Driving Systems: State-of-the-Art and Open Research Challenges

“…Inconsistency Between Methods or Sensors [35], [84], [85] Aims to check the output of N similar methods or sensors, such as object detection & tracking.…”

“…They have tested their method for object detection and vehicle localisation. Additionally, in [85], they have extended their idea in [84], and utilise a Graph Neural Network (GNN) for detecting inconsistencies, i.e. faults.…”

Introspection of 2D Object Detection using Processed Neural Activation Patterns in Automated Driving Systems

“…This formulation enables the system to identify errors in object detection and vehicle localisation with minimal overhead. The idea is extended in [2] where a graph neural network (GNN) detects inconsistencies in the diagnostic graph.…”