Evaluation measures for hierarchical classification: a unified view and novel approaches

Kosmopoulos, Aris; Partalas, Ioannis; Gaussier, Éric; Παλιούρας, Γεώργιος; Androutsopoulos, Ion

doi:10.1007/s10618-014-0382-x

Cited by 134 publications

(88 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The classification performance of the systems was measured using flat and hierarchical evaluation measures (Balikas et al, 2013). The micro F-measure (MiF) and the Lowest Common Ancestor F-measure (LCA-F) were used to choose the winners for each batch (Kosmopoulos et al, 2013).…”

Section: Task 5amentioning

confidence: 99%

Results of the fifth edition of the BioASQ Challenge

Nentidis¹,

Bougiatiotis²,

Krithara³

et al. 2017

BioNLP 2017

Self Cite

View full text Add to dashboard Cite

The goal of the BioASQ challenge is to engage researchers into creating cuttingedge biomedical information systems. Specifically, it aims at the promotion of systems and methodologies that are able to deal with a plethora of different tasks in the biomedical domain. This is achieved through the organization of challenges. The fifth challenge consisted of three tasks: semantic indexing, question answering and a new task on information extraction. In total, 29 teams with more than 95 systems participated in the challenge. Overall, as in previous years, the best systems were able to outperform the strong baselines. This suggests that stateof-the art systems are continuously improving, pushing the frontier of research.

show abstract

Section: Task 5amentioning

confidence: 99%

Results of the fifth edition of the BioASQ Challenge

Nentidis¹,

Bougiatiotis²,

Krithara³

et al. 2017

BioNLP 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…equally, hierarchical measures [35] take into consideration hierarchical distance between the true label and predicted label for evaluating the classifier performance. In general, misclassifications that are closer to the actual class are less severe than misclassifications that are farther from the true class with respect to the hierarchy (e.g., an example from hockey class misclassified as the baseball class is less severe in comparison with the hockey misclassified as cat).…”

Section: Evaluation Metricsmentioning

confidence: 99%

HierFlat: flattened hierarchies for improving top-down hierarchical classification

Naik

Rangwala

2017

Int J Data Sci Anal

View full text Add to dashboard Cite

Large-scale classification of structured data where classes are organized in a hierarchical structure is an important area of research. Top-down approaches that leverage the hierarchy during the learning and prediction phase are efficient for solving large-scale hierarchical classification. However, accuracy of top-down approaches is poor due to error propagation, i.e., prediction errors made at higher levels in the hierarchy cannot be corrected at lower levels. One of the main reasons behind errors at the higher levels is the presence of inconsistent nodes that are introduced due to the arbitrary process of creating these hierarchies by domain experts. In this paper, we propose two different data-driven approaches (local and global) for hierarchical structure modification that identifies and flattens inconsistent nodes present in the hierarchy. Our extensive empirical evaluation of the proposed approaches on several image and text datasets with varying distribution of features, classes and training instances per class shows improved classification performance over competing hierarchical modification approaches. Specifically, we see an improvement up to 7% in Macro-F1 score with our best approach over best top-down baseline. We also show that our modified hierarchy leads to improved classification performance for classes with few training samples in comparison with flat approach. Further, our experimental evaluation shows that combining the prediction from flat and

show abstract

“…In practice, most HMC papers that use the H-loss use the uniform variant (e.g., [19,20,21,22,23,13,24]). Here, we identify two problems with this H-loss variant.…”

Section: Hierarchical Loss Functionsmentioning

confidence: 99%

“…If yes, the thresholds values of t i and its descendants are established accordingly (lines [23][24][25][26]. Once again, this iterative process needs to follow a top-down approach to ensure that the threshold of the…”

Section: Multiple Threshold Selection For the Micro-averaged F-measurementioning

confidence: 99%

Labelling strategies for hierarchical multi-label classification techniques

Triguero

Vens

2016

Pattern Recognition

View full text Add to dashboard Cite

Many hierarchical multi-label classification systems predict a real valued score for every (instance, class) couple, with a higher score reflecting more confidence that the instance belongs to that class. These classifiers leave the conversion of these scores to an actual label set to the user, who applies a cut-off value to the scores. The predictive performance of these classifiers is usually evaluated using threshold independent measures like precision-recall curves. However, several applications require actual label sets, and thus an automatic labelling strategy.In this article, we present and evaluate different alternatives to perform the actual labelling in hierarchical multi-label classification. We investigate the selection of both single and multiple thresholds. Despite the existence of multiple threshold selection strategies in non-hierarchical multi-label classification, they can not be applied directly to the hierarchical context. The proposed strategies are implemented within two main approaches: optimisation of a certain performance measure of interest (such as F-measure or hierarchical loss), and simulating training set properties (such as class distribution or label cardinality) in the predictions. We assess the performance of the proposed labelling schemes on 10 datasets from different application domains. Our results show that selecting multiple thresholds may result in * Corresponding author. Tel : +32(0)9 331 36 93 Fax : +32(0)9 221 76 73Email addresses: Isaac.Triguero@irc.vib-UGent.be (Isaac Triguero), Celine.Vens@kuleuven-kulak.be (Celine Vens) Preprint submitted to ElsevierJanuary 29, 2016 an efficient and effective solution for hierarchical multi-label problems.

show abstract

Evaluation measures for hierarchical classification: a unified view and novel approaches

Cited by 134 publications

References 16 publications

Results of the fifth edition of the BioASQ Challenge

Results of the fifth edition of the BioASQ Challenge

HierFlat: flattened hierarchies for improving top-down hierarchical classification

Labelling strategies for hierarchical multi-label classification techniques

Contact Info

Product

Resources

About