Robust Traffic Sign Recognition Against Camera Failures

Abstract

Failures of the vehicle camera may compromise the correct acquisition of frames, that are subsequently used by autonomous driving tasks. A clear understanding of the behavior of the autonomous driving tasks under such failure conditions, together with strategies to avoid safety is jeopardized, are indeed necessary. This study analyses and improve the performance of Traffic Sign Recognition (TSR) systems for road vehicles under the possible occurrence of camera failures. Our experimental assessment relies on three public datasets, which are commonly used for benchmarking TSR systems. We artificially inject 13 different types of camera failures into the three datasets. Then, we exploit three deep neural networks (DNNs) to classify either a single frame of a traffic sign or a sequence (i.e., a sliding window) of frames. We show that sliding windows significantly improves the robustness of the classifier against altered frames. We confirm our observations through explainable AI, which allows understanding why different classifiers have different performance in case of camera failures.

Extending the function failure modes taxonomy for intelligent systems with embedded AI components

“…including all traffic signs not just speed limit) has been extensively studied in conjunction with autonomous driving (e.g. Atif et al, 2022), being one of the benchmark problem for ML and AI researchers seeking to demonstrate the robustness of their algorithms (Aslansefat et al, 2021). Some automotive manufacturers have attempted to build on the TSR capability with enhanced ADAS features based on TSR information input.…”

“…The task of the embedded AI module is to convert the input from the vision sensor to speed limit (SL) information for the propulsion control module / ACC feature, specifying the upcoming road speed limit (if any changes identified), and distance to the start of enforced speed restriction. At its core, the TSR AI module typically uses ML (convolutional neural networks) to detect, classify and then track through the time sequence of images (Atif et al, 2022). Based on this information, the ACC feature calculates the required control input for the multi-vector vehicle propulsion systems (prime mover, transmission, brakes) to adjust the velocity profile and trajectory such that the desired speed limit is reached with the driveability and comfort expected by the vehicle occupants.…”

Robust Perception and Visual Understanding of Traffic Signs in the Wild

“…Their method, tested on a traffic sign recognition task using self-created 3D driving scenarios, successfully handled Deep Neural Network (DNN) errors associated with in-distribution, out-of-distribution, and adversarial data. A method for traffic sign recognition that is robust against camera failures is proposed in [14]. Through experimental evaluation using three public datasets and artificially injected camera failures, they demonstrate that using a sliding window of frames, rather than a single frame, significantly enhances the robustness of DNN classifiers against compromised frames.…”