A new framework for optimal classifier design

Abstract

The use of alternative measures to evaluate classifier performance is gaining attention, specially for imbalanced problems. However, the use of these measures in the classifier design process is still unsolved. In this work we propose a classifier designed specifically to optimize one of these alternative measures, namely, the so-called F-measure. Nevertheless, the technique is general, and it can be used to optimize other evaluation measures. An algorithm to train the novel classifier is proposed, and the numerical scheme is tested with several databases, showing the optimality and robustness of the presented classifier.

Designing multi-label classifiers that maximize F measures: State of the art

“…variants of the SVM learning algorithm (based on the maximum-margin approach) [18,11,32,24,25], whose objective function is (except for [18]) a convex approximation of an F measure; optimization algorithms whose objective function is a non-convex approximation [9,17,20]; algorithms that tune the decision thresholds of binary classifiers [6,26,27,22,13,12]; and cost-sensitive algorithms [23,13].…”

“…In [9] and [17] learning algorithms that maximize continuous but non-convex approximations of F b were proposed, using numerical optimization techniques.…”

“…To deal with non-convexity, the optimization algorithm was run several times, starting from randomly chosen parameter values. In[17] the class-conditional distribution P(X|Y ) is first estimated, then the TP, FP and FN counts are approximated, for a given discriminant function, by integrating P(X|Y ) in the corresponding decision regions. The parameters of the discriminant function that maximize F b are finally estimated by an optimization algorithm.4.2.…”

Semisupervised Approach to Non Technical Losses Detection

“…As proposed in [14] we attempt to maximize F meassure , and also monitor the values of Precision and Recall. We make the analysis for the default value beta equal to one, which translates in a commitment to equality between the Precision and Recall.…”

Polyps Flagging in Virtual Colonoscopy

“…Once the the candidates detection has been performed, the number of true polyps was much lower than the number of non-polyps patches, a relation on the order of 500:1, which is a significant problem for the learning stage of the classifier, since most classifiers are designed to maximize the accuracy, which is not adequate for imbalanced problems [7]. For instance, if we classify all candidates as "nonpolyps," we would get an accuracy of 99.8% but without detecting any polyps.…”