Benchmarking Safety Monitors for Image Classifiers with Machine Learning

Abstract

High-accurate machine learning (ML) image classifiers cannot guarantee that they will not fail at operation. Thus, their deployment in safety-critical applications such as autonomous vehicles is still an open issue. The use of fault tolerance mechanisms such as safety monitors is a promising direction to keep the system in a safe state despite errors of the ML classifier. As the prediction from the ML is the core information directly impacting safety, many works are focusing on monitoring the ML model itself. Checking the efficiency of such monitors in the context of safety-critical applications is thus a significant challenge. Therefore, this paper aims at establishing a baseline framework for benchmarking monitors for ML image classifiers. Furthermore, we propose a framework covering the entire pipeline, from data generation to evaluation. Our approach measures monitor performance with a broader set of metrics than usually proposed in the literature. Moreover, we benchmark three different monitor approaches in 79 benchmark datasets containing five categories of out-of-distribution data for image classifiers: class novelty, noise, anomalies, distributional shifts, and adversarial attacks. Our results indicate that these monitors are no more accurate than a random monitor. We also release the code of all experiments for reproducibility.

Evaluation of Runtime Monitoring for UAV Emergency Landing

“…A complete literature review about MLRM mechanisms is not the purpose of this paper, but we present a short taxonomy of existing techniques to illustrate the general philosophy inspired from [16]. A first family of approaches predicts misbehavior of a ML model by analyzing its inputs.…”

“…For instance, the work presented in [17] uses a benchmarking methodology based on three datasets, one for training the model, one for tuning the monitor and one for evaluation. Likewise, a complete benchmarking framework, relying on artificial generation of OOD data, is presented in [16]. These approaches present the advantage of using different data sources to fit a monitor and to assess its performance.…”

Benchmarking Safety Monitors for Image Classifiers with Machine Learning

“…In total, we produced 79 benchmark datasets. More details can be found in the results section of our repository [32].…”

Robust Traffic Sign Recognition Against Camera Failures

“…Examples include, but are not limited to: occlusions, shadows, defects of the camera lens, changes in environmental light, raindrops on the camera lens, out-offocus, flare [18], [19]. Therefore, to guarantee safety of the driving task, it is necessary to study the robustness of TSR systems against the aforementioned threats, and develop solutions to tolerate them [20], [21]. This paper accomplishes this tasks, first reviewing the challenges and the state of the art, and then proposing and evaluating alternative solutions.…”