Unified Performance Measure for Binary Classification Problems

Redondo, Ana R.; Navarro, Jorge; Fernández, R.; Diego, Isaac Martín de; Moguerza, Javier M.; Fernández-Muñóz, Juan José

doi:10.1007/978-3-030-62365-4_10

Cited by 13 publications

(7 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, F + 1 is not invariant to class swapping. Furthermore, it is possible to define the F − 1 [22] as the harmonic mean of Specificity and NPV. The F − 1 is a tradeoff between the success of predicting an observation as −1 and the ratio of right predictions in the negative class.…”

Section: Bamentioning

confidence: 99%

“…In [22], a new metric that considers all the elements in the confusion matrix has been recently proposed. The Unified Performance Measure (UPM) is defined as the harmonic mean of F + 1 and F − 1 .…”

Section: Bamentioning

confidence: 99%

“…This performance metric has high values only when the four fundamental metrics, PPV, TPR, PNR, and NPV, also have high values. In addition, UPM is suitable with imbalanced data [22].…”

Section: Bamentioning

confidence: 99%

“…MCC = 0 indicates a random prediction. It has been proven that MCC is not as stable as UPM [22]. The Cohen's Kappa coefficient measures the accordance between the ML classifier and the observed classes as follows:…”

Section: Bamentioning

confidence: 99%

“…The UPM [22] is another performance metric that belongs to the GPS family. The UPM is equals to GPS parameterised with Precision (PPV), Recall (TPR), Specificity (TNR) and Negative Predictive Value (NPV):…”

Section: Binary Classificationmentioning

confidence: 99%

See 4 more Smart Citations

General Performance Score for classification problems

et al. 2022

Self Cite

View full text Add to dashboard Cite

Several performance metrics are currently available to evaluate the performance of Machine Learning (ML) models in classification problems. ML models are usually assessed using a single measure because it facilitates the comparison between several models. However, there is no silver bullet since each performance metric emphasizes a different aspect of the classification. Thus, the choice depends on the particular requirements and characteristics of the problem. An additional problem arises in multi-class classification problems, since most of the well-known metrics are only directly applicable to binary classification problems. In this paper, we propose the General Performance Score (GPS), a methodological approach to build performance metrics for binary and multi-class classification problems. The basic idea behind GPS is to combine a set of individual metrics, penalising low values in any of them. Thus, users can combine several performance metrics that are relevant in the particular problem based on their preferences obtaining a conservative combination. Different GPS-based performance metrics are compared with alternatives in classification problems using real and simulated datasets. The metrics built using the proposed method improve the stability and explainability of the usual performance metrics. Finally, the GPS brings benefits in both new research lines and practical usage, where performance metrics tailored for each particular problem are considered.

show abstract

Section: Bamentioning

confidence: 99%

Section: Bamentioning

confidence: 99%

“…This performance metric has high values only when the four fundamental metrics, PPV, TPR, PNR, and NPV, also have high values. In addition, UPM is suitable with imbalanced data [22].…”

Section: Bamentioning

confidence: 99%

Section: Bamentioning

confidence: 99%

Section: Binary Classificationmentioning

confidence: 99%

See 3 more Smart Citations