Uncertainty-Based Rejection in Machine Learning: Implications for Model Development and Interpretability

Abstract

Uncertainty is present in every single prediction of Machine Learning (ML) models. Uncertainty Quantification (UQ) is arguably relevant, in particular for safety-critical applications. Prior research focused on the development of methods to quantify uncertainty; however, less attention has been given to how to leverage the knowledge of uncertainty in the process of model development. This work focused on applying UQ into practice, closing the gap of its utility in the ML pipeline and giving insights into how UQ is used to improve model development and its interpretability. We identified three main research questions: (1) How can UQ contribute to choosing the most suitable model for a given classification task? (2) Can UQ be used to combine different models in a principled manner? (3) Can visualization techniques improve UQ’s interpretability? These questions are answered by applying several methods to quantify uncertainty in both a simulated dataset and a real-world dataset of Human Activity Recognition (HAR). Our results showed that uncertainty quantification can increase model robustness and interpretability.

Explainability through uncertainty: Trustworthy decision-making with neural networks

“…For predictions with high uncertainty, the observations can be passed on to a human expert for a label. The goal of a classification with rejection system (Mena et al, 2021;Barandas et al, 2022) is to help decide when to stop rejecting the most uncertain observations. The system takes as input the per-observation uncertainty values and outputs three metrics to assist the decision maker in finding the optimal rejection threshold for the task at hand.…”

“…* | x * , θ(t) ) varies greatly for different weights θ(t) . Intuitively, data uncertainty measures uncertainty in the softmax classification on individual samples; model uncertainty measures how much the samples deviate(Hüllermeier & Waegeman, 2021;Barandas et al, 2022).…”

Exploiting Epistemic Uncertainty at Inference Time for Early-Exit Power Saving

“…KNN and distance metrics for anomaly detection have been studied previously [18,5,26], and recently have seen use in OOD and adversarial detection [35,1,39]. Furthermore, the notion of rejecting classification has been studied in depth [14,22,4]. However, this line of work often presents a rejection at the end of inference and so functionally is not dissimilar to methods which calibrate their outputs.…”

Improving Performance and Quantifying Uncertainty of Body-Rocking Detection Using Bayesian Neural Networks

“…Uncertainty quantification can be obtained by means of sampling or direct estimation recovering measures of uncertainty such as entropy, variance, mutual information, etc. Methods based on such approaches have been used for improving classification in activity recognition tasks [32][33][34], as well as eliminating predicted samples with high uncertainty [35]. In the context of multi-instance learning, uncertainty quantification has been used to improve instance level classifiers [36], and to aid active learning scheme to provide different levels of confidence about predicted samples for weak labelers (for the models under training) or strong ones (for the samples that had available labels) [37].…”