LiGCN: Label-interpretable Graph Convolutional Networks for Multi-label Text Classification

Abstract

Multi-label text classification (MLTC) is an attractive and challenging task in natural language processing (NLP). Compared with single-label text classification, MLTC has a wider range of applications in practice. In this paper, we propose a label-interpretable graph convolutional network model to solve the MLTC problem by modeling tokens and labels as nodes in a heterogeneous graph. In this way, we are able to take into account multiple relationships including token-level relationships. Besides, the model allows better interpretability for predicted labels as the token-label edges are exposed. We evaluate our method on four real-world datasets and it achieves competitive scores against selected baseline methods. Specifically, this model achieves a gain of 0.14 on the F1 score in the small label set MLTC, and 0.07 in the large label set scenario.

Diffuser: Efficient Transformers with Multi-Hop Attention Diffusion for Long Sequences

“…Transformers (Vaswani et al 2017) designed for sequential data have revolutionized the field of Natural Language Processing (NLP) (Liu et al 2019;Zhu et al 2020;Li et al 2022), and have recently made tremendous impact in graph learning (Yang et al 2021;Dwivedi and Bresson 2020) and computer vision (Dosovitskiy et al 2020;Huynh 2022). The self-attention used by regular Transformer models comes with a quadratic time and memory complexity O(n 2 ) for input sequence of length n, which prevents the application of Transformers to longer sequences in practical settings with limited computational resources.…”