Input Validation for Neural Networks via Runtime Local Robustness Verification

Liu, Jiangchao; Chen, Liqian; Mine, Antoine; Wang, Ji

doi:10.48550/arxiv.2002.03339

Cited by 1 publication

(2 citation statements)

References 32 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another possible solution based on DNN inputs is to create a radius distance threshold calibrated during the training [4]. The idea is to perturb the DNN inputs, observe the correct answer, and determine how considerable the distance is regarding the DNN decisions.…”

Section: Data-based Monitorsmentioning

confidence: 99%

See 1 more Smart Citation

Benchmarking Safety Monitors for Image Classifiers with Machine Learning

Ferreira,

Arlat,

Guiochet

et al. 2021

Preprint

View full text Add to dashboard Cite

High-accurate machine learning (ML) image classifiers cannot guarantee that they will not fail at operation. Thus, their deployment in safety-critical applications such as autonomous vehicles is still an open issue. The use of fault tolerance mechanisms such as safety monitors is a promising direction to keep the system in a safe state despite errors of the ML classifier. As the prediction from the ML is the core information directly impacting safety, many works are focusing on monitoring the ML model itself. Checking the efficiency of such monitors in the context of safety-critical applications is thus a significant challenge. Therefore, this paper aims at establishing a baseline framework for benchmarking monitors for ML image classifiers. Furthermore, we propose a framework covering the entire pipeline, from data generation to evaluation. Our approach measures monitor performance with a broader set of metrics than usually proposed in the literature. Moreover, we benchmark three different monitor approaches in 79 benchmark datasets containing five categories of out-of-distribution data for image classifiers: class novelty, noise, anomalies, distributional shifts, and adversarial attacks. Our results indicate that these monitors are no more accurate than a random monitor. We also release the code of all experiments for reproducibility.

show abstract

Section: Data-based Monitorsmentioning

confidence: 99%

“…Many recent works focus on monitors dedicated to the ML model surveillance. They broadly fall in three types of monitors: observation of the inputs of the ML model [4], its outputs [5] or from intermediate layers in case of deep neural networks (DNN) [6], [7]. However, they are all based on the exploitation of the training data.…”

Section: Introductionmentioning

confidence: 99%