Failure Prediction with Statistical Guarantees for Vision-Based Robot Control

Farid, Alyaa; Snyder, David B.; Z., Ren, Allen; Majumdar, Anirudha

doi:10.15607/rss.2022.xviii.042

Cited by 6 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are a number of papers that address the specific topic using PAC-based guarantees to generalize error bounds within CPSs. Notable investigations in this area include [22], [19], [2] and [12]. Similar to the objective of this study, the PAC-based guarantees in the aforementioned works correlate the size of the training data to the failure rate with a particular level of confidence.…”

Section: A Pac-based Safety Guaranteesmentioning

confidence: 94%

“…The primary reason for this is the black-boxed nature of DNNs, which, due to the number of training parametres, make it difficult to provide safety assurances within the CPS context [19]. The necessity of these are highlighted by the fact that the performance estimation formed during the training/testing phase of development may be different from the true performance of the system during deployment, oftentimes because of the existence of out-ofdistribution (OOD) data that is unlikely to be present in the training phase [2].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PAC-Based Formal Verification for Out-of-Distribution Data Detection

Prashant

Easwaran

2022

2022 6th International Conference on System Reliability and Safety (ICSRS)

View full text Add to dashboard Cite

Cyber-physical systems (CPS) like autonomous vehicles, that utilize learning components, are often sensitive to noise and out-of-distribution (OOD) instances encountered during runtime. As such, safety critical tasks depend upon OOD detection subsystems in order to restore the CPS to a known state or interrupt execution to prevent safety from being compromised. However, it is difficult to guarantee the performance of OOD detectors as it is difficult to characterize the OOD aspect of an instance, especially in high-dimensional unstructured data.To distinguish between OOD data and data known to the learning component through the training process, an emerging technique is to incorporate variational autoencoders (VAE) within systems and apply classification or anomaly detection techniques on their latent spaces. The rationale for doing so is the reduction of the data domain size through the encoding process, which benefits real-time systems through decreased processing requirements, facilitates feature analysis for unstructured data and allows more explainable techniques to be implemented.This study places probably approximately correct (PAC) based guarantees on OOD detection using the encoding process within VAEs to quantify image features and apply conformal constraints over them. This is used to bound the detection error on unfamiliar instances, , with user-defined confidence, 1 − δ. The approach used in this study is to empirically establish these bounds by sampling the latent probability distribution and evaluating the error with respect to the constraint violations that are encountered. The guarantee is then verified using data generated from CARLA, an open-source driving simulator.

show abstract

Section: A Pac-based Safety Guaranteesmentioning

confidence: 94%

Section: Introductionmentioning

confidence: 99%

PAC-Based Formal Verification for Out-of-Distribution Data Detection

Prashant

Easwaran

2022

2022 6th International Conference on System Reliability and Safety (ICSRS)

View full text Add to dashboard Cite

show abstract

“…Some methods that can provide guarantees come with caveats such as specific architectures [2,34], only providing guarantees in retrospect (i.e. after multiple anomalies have occurred) [3], or on test data that is statistically similar to the training data [35]. We develop an online anomaly detector which is able to provide guarantees for both the false positive rate and the false negative rate.…”

Section: Related Workmentioning

confidence: 99%

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles

Farid¹,

Veer²,

Ivanovic³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In modern autonomy stacks, prediction modules are paramount to planning motions in the presence of other mobile agents. However, failures in prediction modules can mislead the downstream planner into making unsafe decisions. Indeed, the high uncertainty inherent to the task of trajectory forecasting ensures that such mispredictions occur frequently. Motivated by the need to improve safety of autonomous vehicles without compromising on their performance, we develop a probabilistic run-time monitor that detects when a "harmful" prediction failure occurs, i.e., a task-relevant failure detector. We achieve this by propagating trajectory prediction errors to the planning cost to reason about their impact on the AV. Furthermore, our detector comes equipped with performance measures on the false-positive and the false-negative rate and allows for data-free calibration. In our experiments we compared our detector with various others and found that our detector has the highest area under the receiver operator characteristic curve.

show abstract

“…Such out-of-distribution (OOD) approaches can ensure that the model can avoid making predictions when facing an image that does not belong to any of the classes it is trained to predict. OOD detection has been successfully used in several applications, like autonomous systems [ 30 , 31 ], medical diagnosis [ 32 – 34 ], robotics [ 35 , 36 ], and social science [ 37 ], to improve the reliability and safety of systems. OOD detection is desirable in insect pest detection as it allows human intervention in case of uncertain model predictions.…”

Section: Introductionmentioning

confidence: 99%

Out-of-Distribution Detection Algorithms for Robust Insect Classification

Saadati,

Balu,

Chiranjeevi

et al. 2024

Plant Phenomics

View full text Add to dashboard Cite

Plants encounter a variety of beneficial and harmful insects during their growth cycle. Accurate identification (i.e., detecting insects’ presence) and classification (i.e., determining the type or class) of these insect species is critical for implementing prompt and suitable mitigation strategies. Such timely actions carry substantial economic and environmental implications. Deep learning-based approaches have produced models with good insect classification accuracy. Researchers aim to implement identification and classification models in agriculture, facing challenges when input images markedly deviate from the training distribution (e.g., images like vehicles, humans, or a blurred image or insect class that is not yet trained on). Out-of-distribution (OOD) detection algorithms provide an exciting avenue to overcome these challenges as they ensure that a model abstains from making incorrect classification predictions on images that belong to non-insect and/or untrained insect classes. As far as we know, no prior in-depth exploration has been conducted on the role of the OOD detection algorithms in addressing agricultural issues. Here, we generate and evaluate the performance of state-of-the-art OOD algorithms on insect detection classifiers. These algorithms represent a diversity of methods for addressing an OOD problem. Specifically, we focus on extrusive algorithms, i.e., algorithms that wrap around a well-trained classifier without the need for additional co-training. We compared three OOD detection algorithms: (a) maximum softmax probability, which uses the softmax value as a confidence score; (b) Mahalanobis distance (MAH)-based algorithm, which uses a generative classification approach; and (c) energy-based algorithm, which maps the input data to a scalar value, called energy. We performed an extensive series of evaluations of these OOD algorithms across three performance axes: (a) Base model accuracy: How does the accuracy of the classifier impact OOD performance? (b) How does the level of dissimilarity to the domain impact OOD performance? (c) Data imbalance: How sensitive is OOD performance to the imbalance in per-class sample size? Evaluating OOD algorithms across these performance axes provides practical guidelines to ensure the robust performance of well-trained models in the wild, which is a key consideration for agricultural applications. Based on this analysis, we proposed the most effective OOD algorithm as wrapper for the insect classifier with highest accuracy. We presented the results of its OOD detection performance in the paper. Our results indicate that OOD detection algorithms can significantly enhance user trust in insect pest classification by abstaining classification under uncertain conditions.

show abstract

Failure Prediction with Statistical Guarantees for Vision-Based Robot Control

Cited by 6 publications

References 0 publications

PAC-Based Formal Verification for Out-of-Distribution Data Detection

PAC-Based Formal Verification for Out-of-Distribution Data Detection

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles

Out-of-Distribution Detection Algorithms for Robust Insect Classification

Contact Info

Product

Resources

About