Hierarchical Multi-label Text Classification with Horizontal and Vertical Category Correlations

Abstract

Hierarchical multi-label text classification (HMTC) deals with the challenging task where an instance can be assigned to multiple hierarchically structured categories at the same time. The majority of prior studies either focus on reducing the HMTC task into a flat multi-label problem ignoring the vertical category correlations or exploiting the dependencies across different hierarchical levels without considering the horizontal correlations among categories at the same level, which inevitably leads to fundamental information loss. In this paper, we propose a novel HMTC framework that considers both vertical and horizontal category correlations. Specifically, we first design a loosely coupled graph convolutional neural network as the representation extractor to obtain representations for words, documents, and, more importantly, level-wise representations for categories, which are not considered in previous works. Then, the learned category representations are adopted to capture the vertical dependencies among levels of category hierarchy and model the horizontal correlations. Finally, based on the document embeddings and category embeddings, we design a hybrid algorithm to predict the categories of the entire hierarchical structure. Extensive experiments conducted on real-world HMTC datasets validate the effectiveness of the proposed framework with significant improvements over the baselines.

Hierarchical Text Classification and Its Foundations: A Review of Current Research

“…The latter can be found in Tables 9 and 10, while the results over the individual splits are reported in Table 11. [149] 0.810 0.533 --HiLAP [99] 0.833 0.601 --HiAGM-TP [97] 0.840 0.634 0.858 0.803 RLHR [167] --0.785 0.792 HCSM [168] 0.858 0.609 0.921 0.807 HiMatch [124] 0.847 0.641 0.862 0.805 HIDDEN [169] 0.793 0.473 --HE-AGCRCNN [170] 0.778 0.513 --HVHMC [171] --0.743 -SASF [126] --0.867 0.811 HTCInfoMax [177] 0.835 0.627 0.856 0.800 PAAM-HiA-T5 [178] 0.872 0.700 0.904 0.816 HPT [136] 0.873 0.695 0.872 0.819 HGCLR [91] 0.865 0.683 0.871 0.812 Seq2Tree [93] 0.869 0.700 0.872 0.825 HBGL [180] 0.872 0.711 0.874 0.820 P-tuning v2 (SPP-tuning) [138] --0.875 0.800 LD-GGNN [186] 0.842 0.641 0.851 0.805 LSE-HiAGM [123] 0.839 0.646 0.860 0.800 Seq2Label [121] 0.874 0.706 0.873 0.819 HTC-CLIP [190] --0.879 0.816 GACaps [191] 0.868 0.698 0.876 0.828 HiDEC [194] 0.855 0.651 --UMP-MG [129] --0.859 0.813 LED [196] 0.883 0.697 0.870 0.813 (HGCLR-based + aug) [198] 0.862 0.679 0.874 0.821 K-HTC [135] --0.873 0.817 HiTIN (BERT) [200] 0.867 0.699 0.872 0.816 HierVerb (few-shot) [137] 0 The standard deviation over the 2 repetitions of 3-fold cross-validation is reported in brackets.…”

“…First, a considerable number of works on HTC have recently been taking into consideration the semantics of labels to improve classification performance [110,123,124,171,179]. Some of them devise a way to obtain label embeddings and then use this information to produce label-aware document embeddings.…”

“…Despite this approach producing good results in [124], the usage of contextualized models to produce embeddings of small fragments of text without enough "context" is not completely justified according, to the previous literature [1], or at least it does not produce a proper semantic embedding. In a similar vein, some methods attempt to compensate for the lack of external metadata describing the label semantics by computing latent representation using auto-encoders [179] and GCNs [123,170,171].…”

Attention Mechanisms in Clinical Text Classification: A Comparative Evaluation

“…Others use graph neural networks (GNN) to learn the hierarchical relationships of the label space [22], [23]. Xu et al [24] combined label correlation and a GNN to learn a precise representation of the hierarchy. Shen et al [25] instead used weak supervision to create a hierarchical structure of the label space.…”

Hierarchical Label Generation for Text Classification

“…Despite the previous methods being successful, their approaches classify labels sequentially by choosing them from the predefined label set in the training dataset. It is still an open problem for real-world applications to handle unseen labels that do not appear in the pre-defined label set from the training dataset (Banerjee et al, 2019;Aly et al, 2019;Xu et al, 2021). Due to severe deficiencies in annotating data for labels in a hierarchy and handling unseen labels for real-world applications , we need a general modeling framework for handling unseen labels while explicitly incorporating a label hierarchy to overcome the restriction of the pre-defined label set for the development of real-world text classification applications.…”