Where Does Trust Break Down? A Quantitative Trust Analysis of Deep Neural Networks via Trust Matrix and Conditional Trust Densities

Abstract

With tremendous rise in deep learning adoption comes questions about the trustworthiness of the deep neural networks that power a variety of applications. In this work, we introduce the concept of trust matrix, a novel trust quantification strategy that leverages the recently introduced question-answer trust metric by Wong et al. to provide deeper, more detailed insights into where trust breaks down for a given deep neural network given a set of questions. More specifically, a trust matrix defines the expected question-answer trust for a given actor-oracle answer scenario, allowing one to quickly spot areas of low trust that needs to be addressed in order to improve the trustworthiness of a deep neural network. We further extend the concept of trust densities with the notion of conditional trust densities.

Video-based formative and summative assessment of surgical tasks using deep learning

“…The VBA-Net has robust trustworthiness with NTS values of 0.926 and 0.868 for the passing and failing classes. Moreover, for both the classes, the conditional NTS is above 0.9 when the prediction is true and around 0.3 when the prediction is false, implying that the VBA-Net has strong confidence in true predictions with low uncertainty while it can benefit from additional data for both classes 35 .…”

“…The model's trustworthiness is analyzed in a single training session via trustworthiness metrics 35,37 . Figure 2c shows the trust spectrum accompanied by the NetTrustScore (NTS).…”

“…In contrast, ρ was used for OSATS (ρ OSATS ) and GRS (ρ GRS ) score predictions where ρ OSATS was the mean value of ρs for every six OSATS subscores 9,13,34 www.nature.com/scientificreports/ Trustworthiness. Besides the commonly used metrics, we utilized the recently proposed trustworthiness metrics 35,36 , i.e., question-answer trust, trust density, conditional trust density, trust spectrum, and NetTrust-Score (NTS), to assess the reliability of the VBA-Net on the classification results. In this concept, the Softmax probability is associated with confidence, C(y|x), and a model, M, is trustworthy when a true prediction is accompanied by stronger Softmax and vice versa.…”

A Trust-Based Methodology to Evaluate Deep Learning Models for Automatic Diagnosis of Ocular Toxoplasmosis from Fundus Images

“…The general purpose trust score proposed by Wong et al [24] and extended by Hryniowski et al [25] defines trust based on the answer to two questions: (1) How much trust do we have in a model that gives wrong answers with great confidence? and (2) How much trust do we have in a model that gives right answers hesitantly?…”

“…A general-purpose trust metric was proposed by Wong et al [24] and extended by Hryniowski et al [25]. They were experimentally tested with Imagenet with insightful results.…”