Designing multi-label classifiers that maximize F measures: State of the art

Abstract

Multi-label classification problems usually occur in tasks related to information retrieval, like text and image annotation, and are receiving increasing attention from the machine learning and pattern recognition fields. One of the main issues under investigation is the development of classification algorithms capable of maximizing specific accuracy measures based on precision and recall. We focus on the widely used F measure, defined for binary, single-label problems as the weighted harmonic mean of precision and recall, and later extended to multi-label problems in three ways: macro-averaged, micro-averaged and instance-wise. In this paper we give a comprehensive survey of theoretical results and algorithms aimed at maximizing F measures. We subdivide it according to the two main existing approaches: empirical utility maximization, and decision-theoretic. Under the former approach, we also derive the optimal (Bayes) classifier at the population level for the instance-wise and micro-averaged F, extending recent results about the single-label F. In a companion paper we shall focus on the micro-averaged F measure, for which relatively fewer solutions exist, and shall develop novel maximization algorithms under both approaches

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

“…Nowadays, the F 1 measure is widely used in most application areas of machine learning, not only in the binary scenario, but also in multiclass cases. In multiclass cases, researchers can employ the F 1 micro/macro averaging procedure [55–60], which can be even targeted for ad-hoc optimization [61].…”

System Architecture for Real-Time Surface Inspection Using Multiple UAVs

“…Specifically, Otsu Fig. 13: Errors between the planned and flown paths The comparison results can be quantitatively evaluated via the F -measure, a compromise between recall and precision [43]. Let t p and t n be the correctly reported positive and negative results, whereas f p and f n be the falsely reported positive and negative results.…”

Consistent prediction of GO protein localization

“…Note that since our GO-CC predictions follow from MAP estimates, we do not expect they are able to optimize more elaborate performance metrics like the F-score. We note, however, that the design of optimal F-score classification algorithms remain a challenging computational problem even for the prediction of flat multiclass/multilabel categories 74 .…”