A unified view of class-selection with probabilistic classifiers

Abstract

Abstract. The possibility of selecting a subset of classes instead of one unique class for assignation is of great interest in many decision making systems. Selecting a subset of classes instead of singleton allows to reduce the error rate and to propose a reduced set to another classifier or an expert. This second step provides additional information, and therefore increases the quality of the result. In this paper, a unified view of the problem of class-selection with probabilistic classifiers is presented. The proposed framework, based on the evaluation of the probabilistic equivalence, allows to retrieve class-selective frameworks that have been proposed in the literature. We also describe an approach in which the decision rules are compared by the help of a normalized area under the error/selection curve. It allows to get a relative independence of the performance of a classifier without reject option, and thus a reliable class-selection decision rule evaluation. The power of this generic proposition is demonstrated by evaluating and comparing it to several state of the art methods on nine real world data sets, and four different probabilistic classifiers.

Performance visualization spaces for classification with rejection option

“…The classification with reject option consists to train a model, called reject classifier, that is able to answer "I do not know" when the confidence in its prediction is low [2] [19]. Also called abstaining classifier [22], selective classification [8] or reject classification [25], this type of model receives an increasing interest in the machine learning community because of its application in many real world classification tasks.…”