Towards Better Confidence Estimation for Neural Models

Vasudevan, V.; Sethy, Abhinav; Ghias, Alireza Roshan

doi:10.1109/icassp.2019.8683359

Cited by 23 publications

(15 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These probabilities can be already interpreted as a prediction of the data uncertainty. However, it is widely discussed that neural networks are often over-confident and the softmax output is often poorly calibrated, leading to inaccurate uncertainty estimates [95], [67], [44], [92]. Furthermore, the softmax output cannot be associated with model uncertainty.…”

Section: Uncertainty Estimationmentioning

confidence: 99%

A Survey of Uncertainty in Deep Neural Networks

Gawlikowski¹,

Tassi²,

Ali³

et al. 2021

Preprint

163

158

View full text Add to dashboard Cite

and safety-critical real world applications remains limited. The main factors responsible for this limitation are• the lack of expressiveness and transparency of a deep neural network's inference model, which makes it difficult to trust their outcomes [2],• the inability to distinguish between in-domain and out-ofdomain samples [11], [12] and the sensitivity to domain shifts [13],• the inability to provide reliable uncertainty estimates for a deep neural network's decision [14] and frequently occurring overconfident predictions [15], [16],• the sensitivity to adversarial attacks that make deep neural networks vulnerable for sabotage [17], [18], [19].

show abstract

Section: Uncertainty Estimationmentioning

confidence: 99%

A Survey of Uncertainty in Deep Neural Networks

Gawlikowski¹,

Tassi²,

Ali³

et al. 2021

Preprint

163

158

View full text Add to dashboard Cite

show abstract

“…For simple classifiers, measuring softmax probabilities, or information theoretic metrics such as entropy, and mutual information [37] may suffice. For more complex networks such as those than operate on a SDC, softmax probabilities and entropy are known to be unreliable confidence estimators [52]. Moreover, white-box metrics require a transparent access to the network, and substantial domain-knowledge for the creation of nontrivial probabilistic models that approximate the network's uncertainty.…”

Section: Introductionmentioning

confidence: 99%

Misbehaviour prediction for autonomous driving systems

Stocco

Weiß

Calzana

et al. 2020

Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering

125

131

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) are the core component of modern autonomous driving systems. To date, it is still unrealistic that a DNN will generalize correctly to all driving conditions. Current testing techniques consist of offline solutions that identify adversarial or corner cases for improving the training phase. In this paper, we address the problem of estimating the confidence of DNNs in response to unexpected execution contexts with the purpose of predicting potential safety-critical misbehaviours and enabling online healing of DNN-based vehicles. Our approach SelfOracle is based on a novel concept of self-assessment oracle, which monitors the DNN confidence at runtime, to predict unsupported driving scenarios in advance. SelfOracle uses autoencoderand time series-based anomaly detection to reconstruct the driving scenarios seen by the car, and to determine the confidence boundary between normal and unsupported conditions. In our empirical assessment, we evaluated the effectiveness of different variants of SelfOracle at predicting injected anomalous driving contexts, using DNN models and simulation environment from Udacity. Results show that, overall, SelfOracle can predict 77% misbehaviours, up to six seconds in advance, outperforming the online input validation approach of DeepRoad. CCS CONCEPTS • Software and its engineering → Software testing and debugging.

show abstract

“…Trustworthiness of a simple DNN can be measured with softmax probabilities [13], or information theoretic metrics, such as entropy [14] and mutual information [15]. However, in complex DNNs with many layers and neurons, softmax probabilities and entropy are unreliable confidence estimators of the prediction [16], [17]. Even for abnormal samples, DNNs may still produce overconfident posterior probabilities.…”

Section: Introductionmentioning

confidence: 99%

Self-Checking Deep Neural Networks in Deployment

Xiao

Beschastnikh

Rosenblum

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

The widespread adoption of Deep Neural Networks (DNNs) in important domains raises questions about the trustworthiness of DNN outputs. Even a highly accurate DNN will make mistakes some of the time, and in settings like self-driving vehicles these mistakes must be quickly detected and properly dealt with in deployment.Just as our community has developed effective techniques and mechanisms to monitor and check programmed components, we believe it is now necessary to do the same for DNNs. In this paper we present DNN self-checking as a process by which internal DNN layer features are used to check DNN predictions. We detail SelfChecker, a self-checking system that monitors DNN outputs and triggers an alarm if the internal layer features of the model are inconsistent with the final prediction. SelfChecker also provides advice in the form of an alternative prediction.We evaluated SelfChecker on four popular image datasets and three DNN models and found that SelfChecker triggers correct alarms on 60.56% of wrong DNN predictions, and false alarms on 2.04% of correct DNN predictions. This is a substantial improvement over prior work (SELFORACLE, DISSECTOR, and ConfidNet). In experiments with self-driving car scenarios, SelfChecker triggers more correct alarms than SELFORACLE for two DNN models (DAVE-2 and Chauffeur) with comparable false alarms. Our implementation is available as open source.

show abstract

Towards Better Confidence Estimation for Neural Models

Cited by 23 publications

References 9 publications

A Survey of Uncertainty in Deep Neural Networks

A Survey of Uncertainty in Deep Neural Networks

Misbehaviour prediction for autonomous driving systems

Self-Checking Deep Neural Networks in Deployment

Contact Info

Product

Resources

About