Facility Locations Utility for Uncovering Classifier Overconfidence

Abstract

Assessing the predictive accuracy of black box classifiers is challenging in the absence of labeled test datasets. In these scenarios we may need to rely on a human oracle to evaluate individual predictions; presenting the challenge to create query algorithms to guide the search for points that provide the most information about the classifier's predictive characteristics. Previous works have focused on developing utility models and query algorithms for discovering unknown unknowns -misclassifications with a predictive confidence above some arbitrary threshold. However, if misclassifications occur at the rate reflected by the confidence values, then these search methods reveal nothing more than a proper assessment of predictive certainty. We are unable to properly mitigate the risks associated with model deficiency when the model's confidence in prediction exceeds the actual model accuracy. We propose a facility locations utility model and corresponding greedy query algorithm that instead searches for overconfident unknown unknowns. Through robust empirical experiments we demonstrate that the greedy query algorithm with the facility locations utility model consistently results in oracle queries with superior performance in discovering overconfident unknown unknowns than previous methods.

Generalized Adversarial Distances to Efficiently Discover Classifier Errors

“…Algorithmic approaches for finding high-confidence errors from machine learning classifiers mainly consist of the following components: 1) a utility function to measure the usefulness of queried points, 2) some search strategy to help maximize the utility function [4]- [6] In Lakkaraju (2017) [4], they defined a utility function that gave a uniform value for each error discovered above some threshold (65% for binary classification), and applied a penalty associated with the cost of the human providing the true label for the queried points. The suggested search was then to cluster data points from the unlabeled evaluation dataset using some feature space (which, in the case of black-box classifiers, may be different than the original feature space), and perform multiarmed bandit sampling to maximize the utility.…”

“…Meaning, the methods could be discovering errors accidentally, not by discovering commonalities between errors to increase the rate of error discovery. More recent works have instead focused on discovering classification errors at rates greater than expected, to encourage search methods that discover something about a model's weaknesses to increase the rate of error discovery [6], [7]. Of particular interest to this work is a search technique that leverages adversarial machine learning to point users towards high confidence errors.…”

Harnessing Adversarial Distances to Discover High-Confidence Errors

“…Maurer and Bennette (2019) [7] present an extension to [4] and [5] that identifies the flaw of valuing error discovery at the rate expected given model confidence. Meaning, the work identifies the fact that errors should be expected for confidence levels below 100%.…”

“…We consider the problem of discovering high-confidence errors at rates greater than what a model's confidence would suggest, which was recently introduced by [7]. Given a blackbox classifier, M , with M (x) = (ŷ x , px ), where x is an instance from an unlabeled evaluation set X, ŷx is the model's prediction, px is the model's confidence, and y x is the true label assigned by some oracle, the task is to find a query set of data points, Q ⊆ X, that maximize the Standardized Discovery Ratio (SDR).…”

Overprecision is a property of thinking systems

“…Artificially intelligent agents are also overconfident (Tesauro, Gondek, Lenchner, Fan, & Prager, 2013), especially when assumptions underlying their models are violated (Attenberg, Ipeirotis, & Provost, 2011). The failure to account for unknown unknowns represents an important reason why they are overconfident (Maurer & Bennette, 2018). Any set of training data, however large, will be smaller than the universe of potential problems.…”