Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs

Lu, Jueqing; Du, Lan; Liu, Ming; Dipnall, Joanna F.

doi:10.48550/arxiv.2010.07459

Cited by 2 publications

(3 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Chalkidis et al (2020Chalkidis et al ( , 2019; Xie et al (2019) adopted Label-Wise Attention Networks to encourage interactions between text and labels. Rios and Kavuluru (2018); Lu et al (2020) used Graph Neural Networks to capture the structural information in the label hierarchy. However, few existing works investigate the effectiveness of pretrained models on the ZS-MTC task, despite pretrained models being effective as matching models for many natural language processing tasks (Ma et al, 2019;Qiao et al, 2019;Nogueira et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

“…The existing zero-shot learning for multi-label text classification (ZS-MTC) mostly learns a matching model between the feature space of text and the label space (Ye et al, 2020). In order to learn effective representations for labels, a majority of existing work incorporates label hierarchies via a label encoder designed as Graph Neural Networks (GNNs) that can aggregate the neighboring information for labels (Chalkidis et al, 2020;Lu et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning

Liu

Zhang

Yin

et al. 2021

Preprint

View full text Add to dashboard Cite

Exploiting label hierarchies has become a promising approach to tackling the zero-shot multi-label text classification (ZS-MTC) problem. Conventional methods aim to learn a matching model between text and labels, using a graph encoder to incorporate label hierarchies to obtain effective label representations (Rios and Kavuluru, 2018). More recently, pretrained models like BERT (Devlin et al., 2018) have been used to convert classification tasks into a textual entailment task (Yin et al., 2019). This approach is naturally suitable for the ZS-MTC task. However, pretrained models are underexplored in the existing work because they do not generate individual vector representations for text or labels, making it unintuitive to combine them with conventional graph encoding methods. In this paper, we explore to improve pretrained models with label hierarchies on the ZS-MTC task. We propose a Reinforced Label Hierarchy Reasoning (RLHR) approach to encourage interdependence among labels in the hierarchies during training. Meanwhile, to overcome the weakness of flat predictions, we design a rollback algorithm that can remove logical errors from predictions during inference. Experimental results on three reallife datasets show that our approach achieves better performance and outperforms previous non-pretrained methods on the ZS-MTC task.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning

Liu

Zhang

Yin

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This is because auto-tagging has a potentially very large label space, ranging from subject topics to knowledge components (KC) (Zhang et al, 2015;Koedinger et al, 2012;Mohania et al, 2021;Viswanathan et al, 2022). The resulting data scarcity decreases performance on rare labels during training (Chalkidis et al, 2020;Lu et al, 2020;Snell et al, 2017;Choi et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

A Study on the Perception of SNS Users on Artificial Intelligence Design Using Text Mining

Lee¹

2023

kmms

View full text Add to dashboard Cite

Text classification in education, usually called auto-tagging, is the automated process of assigning relevant tags to educational content, such as questions and textbooks. However, auto-tagging suffers from a data scarcity problem, which stems from two major challenges: 1) it possesses a large tag space and 2) it is multi-label. Though a retrieval approach is reportedly good at low-resource scenarios, there have been fewer efforts to directly address the data scarcity problem. To mitigate these issues, here we propose a novel retrieval approach CEAA that provides effective learning in educational text classification. Our main contributions are as follows: 1) we leverage transfer learning from question-answering datasets, and 2) we propose a simple but effective data augmentation method introducing cross-encoder style texts to a bi-encoder architecture for more efficient inference. An extensive set of experiments shows that our proposed method is effective in multi-label scenarios and low-resource tags compared to state-of-the-art models.

show abstract

Multi-label Few/Zero-shot Learning with Knowledge Aggregated from Multiple Label Graphs

Cited by 2 publications

References 35 publications

Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning

Improving Pretrained Models for Zero-shot Multi-label Text Classification through Reinforced Label Hierarchy Reasoning

A Study on the Perception of SNS Users on Artificial Intelligence Design Using Text Mining

Contact Info

Product

Resources

About