An experimental comparison of performance measures for classification

Assessments Metrics for Multi-class Imbalance Learning: A Preliminary Study

“…However, in general terms the global measures show similar ranks. This is consistent with the presented in Reference [8]. Table 2 shows some interesting results.…”

“…Furthermore, a quantitative representation of a ROC curve is the area under it, which is known as AUC [9]. The AUC measure has been adapted at multi-class problems [8] and can be defined as follow.…”

“…On two-class imbalance problems, global metrics to assessments of the classifier performance have been amply studied [7], and in the multi-class imbalance problems some of them have been modified to be adapted at the multi-class imbalance context [8], for example the geometric mean, F-measure or measures of the area under curve family.…”

Personal comfort models: Predicting individuals' thermal preference using occupant heating and cooling behavior and machine learning

“…AUC can vary between 0 and 1; AUC = 0.5 denotes random guessing while 1.0 indicates perfect accuracy. We use the "one versus the rest" method [68] to extend binary ROC into the three-class classification problem of thermal preference. The overall performance of a thermal preference classifier is computed by averaging AUC of the ROC curves for all three classes.…”

Classification Rule Mining with Iterated Greedy

“…-Rule accuracy Improvement (RI): Rule accuracy is defined as the percentage of patterns correctly classified by the rule, i.e., those covered by its antecedent that belong to the predicted class (true positives -TP) and those not covered that belong to a different class (true negatives -TN) (Equation 1; NP is the number of patterns). This metric is the common accuracy measure for binary classification [16,17]. According to this, new conditions result in an accuracy improvement of the rule, but not necessarily of the global system.…”