Deep Learning for Extreme Multi-label Text Classification

Liu, Jingzhou; Chang, Wei‐Cheng; Wu, Yuexin; Yang, Yiming

doi:10.1145/3077136.3080834

Cited by 528 publications

(376 citation statements)

References 22 publications

Supporting

Mentioning

374

Contrasting

Unclassified

Order By: Relevance

“…• Sequence, Graph and N-gram based models These types of models first transform the text dataset into sequences of words, the graph of words or N-grams features, later apply different types of deep learning models on those features including CNN (Kim, 2014b), CNN-RNN (Chen et al, 2017), RCNN (Lai et al, 2015), DCNN (Schwenk et al, 2017), XML-CNN (Liu et al, 2017), HR-DGCNN (Peng et al, 2018), Hierarchical LSTM (HLSTM) (Chen et al, 2016), multi-label classification approach based on a conditional cyclic directed graphical model (CDN-SVM) (Guo and Gu, 2011), Hierarchical Attention Network (HAN) (Yang et al, 2016) and Bi-directional Block Self-Attention Network (Bi-BloSAN) (Shen et al, 2018) etc. for the multilabel classification task For example, Hierarchical Attention Networks for Document Classification (HAN) uses a GRU grating mechanism to encode the sequences and apply word and sentence level attention on those sequences for document classification.…”

Section: Comparison Of Methodsmentioning

confidence: 99%

See 1 more Smart Citation

MAGNET: Multi-Label Text Classification using Attention-based Graph Neural Network

Pal¹,

Selvakumar²,

Sankarasubbu³

2020

Proceedings of the 12th International Conference on Agents and Artificial Intelligence

View full text Add to dashboard Cite

In Multi-Label Text Classification (MLTC), one sample can belong to more than one class. It is observed that most MLTC tasks, there are dependencies or correlations among labels. Existing methods tend to ignore the relationship among labels. In this paper, a graph attention network-based model is proposed to capture the attentive dependency structure among the labels. The graph attention network uses a feature matrix and a correlation matrix to capture and explore the crucial dependencies between the labels and generate classifiers for the task. The generated classifiers are applied to sentence feature vectors obtained from the text feature extraction network(BiLSTM) to enable end-to-end training. Attention allows the system to assign different weights to neighbor nodes per label, thus allowing it to learn the dependencies among labels implicitly. The results of the proposed model are validated on five real-world MLTC datasets. The proposed model achieves similar or better performance compared to the previous state-of-the-art models.

show abstract

Section: Comparison Of Methodsmentioning

confidence: 99%

“…Many CNN based model, RCNN (Lai et al, 2015), Ensemble method of CNN and RNN by Chen et al (2017), XML-CNN (Liu et al, 2017), CNN (Kim, 2014a) and TEXTCNN (Kim, 2014a) have been proposed to solve the MLTC task. However, they neglect the correlations between labels.…”

Section: Neural Network Modelsmentioning

confidence: 99%

MAGNET: Multi-Label Text Classification using Attention-based Graph Neural Network

Pal¹,

Selvakumar²,

Sankarasubbu³

2020

Proceedings of the 12th International Conference on Agents and Artificial Intelligence

View full text Add to dashboard Cite

show abstract

“…An intuitively reasonable objective for multilabel classification is rank loss [50], which minimizes the number of mis-ordered pairs of relevant and irrelevant labels. There is a propensity that we aim to tag relevant labels with high scores than irrelevant labels [23]. However, in feedforward neural network architecture, the rank loss has shown to be inferior to binary cross-entropy loss over sigmoid activation when applied to multi-label classification datasets [32], especially the datasets in the textual domain.…”

Section: Hiepar: Hierarchical and Transparent Representationmentioning

confidence: 99%

A Multi-Label Classification Method Using a Hierarchical and Transparent Representation for Paper-Reviewer Recommendation

Zhang

Zhao

Duan

et al. 2020

ACM Trans. Inf. Syst.

View full text Add to dashboard Cite

Paper-reviewer recommendation task is of significant academic importance for conference chairs and journal editors. It aims to recommend appropriate experts in a discipline to comment on the quality of papers of others in that discipline. How to effectively and accurately recommend reviewers for the submitted papers is a meaningful and still tough task. Generally, the relationship between a paper and a reviewer often depends on the semantic expressions of them. Creating a more expressive representation can make the peer-review process more robust and less arbitrary. So, the representations of a paper and a reviewer are very important for the paper-reviewer recommendation. Actually, a reviewer or a paper often belongs to multiple research fields, which increases difficulty in paper-reviewer recommendation. In this paper, we propose a Multi-Label Classification method using a HIErarchical and transPArent Representation named Hiepar-MLC. Firstly, we introduce a HIErarchical and transPArent Representation (Hiepar) to express the semantic information of the reviewer and the paper. Hiepar is learned from a two-level bidirectional gated recurrent unit based network applying the attention mechanism. It is capable of capturing the two-level hierarchical information (word-sentence-document) and highlighting the elements in reviewers or papers to support the labels. This word-sentence-document information mirrors the hierarchical structure of a reviewer or a paper and captures the exact semantics of them. Then we transform the paper-reviewer recommendation problem into a Multi-Label Classification (MLC) issue, whose multiple research labels exactly guide the learning process. It's flexible that we can select any multi-label classification method to solve the paper-reviewer recommendation problem. Further, we propose a simple multi-label based reviewer assignment (MLBRA) strategy to select the appropriate reviewers. It's interesting that we also explore the paper-reviewer recommendation in the coarse-grained granularity. Extensive experiments on the real-world dataset consisting of the papers in the ACM Digital Library show that MLC-Hiepar achieves better label prediction performance than the existing representation alternatives. Also, with the MLBRA strategy, we show the effectiveness and the feasibility of our transformation from paper-reviewer recommendation to multi-label classification. . 2019. A multi-label classification method using a hierarchical and transparent representation for paper-reviewer recommendation.

show abstract

“…However, this mechanism could be problematic when the utterance is long. We experimented with Dynamic k-Max Pooling [12] to pool the most powerful features from p sub-sequences of an utterance with m words. This pooling scheme naturally deals with variable utterance length.…”

Section: Dynamic K-max Poolingmentioning

confidence: 99%

Modeling Long-Range Context for Concurrent Dialogue Acts Recognition

Peng

Yang

2019

Proceedings of the 28th ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

In dialogues, an utterance is a chain of consecutive sentences produced by one speaker which ranges from a short sentence to a thousand-word post. When studying dialogues at the utterance level, it is not uncommon that an utterance would serve multiple functions. For instance, "Thank you. It works great. " expresses both gratitude and positive feedback in the same utterance. Multiple dialogue acts (DA) for one utterance breeds complex dependencies across dialogue turns. Therefore, DA recognition challenges a model's predictive power over long utterances and complex DA context. We term this problem Concurrent Dialogue Acts (CDA) recognition. Previous work on DA recognition either assumes one DA per utterance or fails to realize the sequential nature of dialogues. In this paper, we present an adapted Convolutional Recurrent Neural Network (CRNN) which models the interactions between utterances of long-range context. Our model significantly outperforms existing work on CDA recognition on a tech forum dataset.

show abstract

Deep Learning for Extreme Multi-label Text Classification

Cited by 528 publications

References 22 publications

MAGNET: Multi-Label Text Classification using Attention-based Graph Neural Network

MAGNET: Multi-Label Text Classification using Attention-based Graph Neural Network

A Multi-Label Classification Method Using a Hierarchical and Transparent Representation for Paper-Reviewer Recommendation

Modeling Long-Range Context for Concurrent Dialogue Acts Recognition

Contact Info

Product

Resources

About