Adaptive Threshold Estimation via Extreme Value Theory

IEEE Trans. Pattern Anal. Mach. Intell.

Rocha

Micheals

et al. 2011

185

125

Abstract-In this paper, we define meta-recognition, a performance prediction method for recognition algorithms, and examine the theoretical basis for its post-recognition score analysis form through the use of the statistical extreme value theory (EVT). The ability to predict the performance of a recognition system based on its outputs for each match instance is desirable for a number of important reasons, including automatic threshold selection for determining matches and non-matches, and automatic algorithm selection or weighting for multi-algorithm fusion. The emerging body of literature on post-recognition score analysis has been largely constrained to biometrics, where the analysis has been shown to successfully complement or replace image quality metrics as a predictor. We develop a new statistical predictor based upon the Weibull distribution, which produces accurate results on a per instance recognition basis across different recognition problems. Experimental results are provided for two different face recognition algorithms, a fingerprint recognition algorithm, a SIFT-based object recognition system, and a content-based image retrieval system.

Section: B the Theoretical Basis Of Meta-recognitionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Meta-Recognition: The Theory and Practice of Recognition Score Analysis

IEEE Trans. Pattern Anal. Mach. Intell.

Rocha

Micheals

et al. 2011

185

125

“…Recent work [13] looking at verification score spaces relies on this intuition, but does not explain why extrema value theory applies to the tails of their score distributions. Just being in the tail is not sufficient to make this an extreme value problem, as one can take the top N samples from any particular distribution D, which by definition fit distribution D and not any other distribution.…”

Section: Statistical Extreme Value Theorymentioning

confidence: 99%

Robust Fusion: Extreme Value Theory for Recognition Score Normalization

Computer Vision – ECCV 2010

Rocha

Micheals

et al. 2010

Abstract. Recognition problems in computer vision often benefit from a fusion of different algorithms and/or sensors, with score level fusion being among the most widely used fusion approaches. Choosing an appropriate score normalization technique before fusion is a fundamentally difficult problem because of the disparate nature of the underlying distributions of scores for different sources of data. Further complications are introduced when one or more fusion inputs outright fail or have adversarial inputs, which we find in the fields of biometrics and forgery detection. Ideally a score normalization should be robust to model assumptions, modeling errors, and parameter estimation errors, as well as robust to algorithm failure. In this paper, we introduce the w-score, a new technique for robust recognition score normalization. We do not assume a match or non-match distribution, but instead suggest that the top scores of a recognition system's non-match scores follow the statistical Extreme Value Theory, and show how to use that to provide consistent robust normalization with a strong statistical basis.

“…For recognition problems in computer vision, EVT has been demonstrated to be a powerful explanatory theory [43] and an effective tool for statistical modeling [7,22,21], including fitting probability estimators [44,42,41]. The most relevant work in EVT modeling is the multi-attribute spaces approach of Scheirer et al [42], which applies EVT calibration over binary classifiers for visual attribute assignment.…”

Section: Related Workmentioning

confidence: 99%

Multi-class Open Set Recognition Using Probability of Inclusion

Jain

Lecture Notes in Computer Science

Boult

2014

277

148

Abstract. The perceived success of recent visual recognition approaches has largely been derived from their performance on classification tasks, where all possible classes are known at training time. But what about open set problems, where unknown classes appear at test time? Intuitively, if we could accurately model just the positive data for any known class without overfitting, we could reject the large set of unknown classes even under an assumption of incomplete class knowledge. In this paper, we formulate the problem as one of modeling positive training data at the decision boundary, where we can invoke the statistical extreme value theory. A new algorithm called the PI -SVM is introduced for estimating the unnormalized posterior probability of class inclusion.