MsCoa: Multi-Step Co-Attention Model for Multi-Label Classification

Abstract

Multi-label text classification (MLC) task, as one of the sub-tasks of natural language processing, has broad application prospects. On the basis of studying the previous research work, this research takes the relationship among text information, leading label information and predictive label information as the frame and analyzes the information loss of original text and leading label, decoding error accumulation. We propose an improved multi-step multi-classification model to mitigate the phenomenon of error prediction, label repetition and error accumulation. The model uses multi-step and multi-classification task to complete multi-label prediction. It uses the leading label and the original text as input, and the next to-be-predicted label as output. The co-attention mechanism is operated between the original text and the leading label. The attention of the original text to the leading label is helpful to filter out the error accumulation problem caused by the error prediction. The features is combined in a manner of difference and concatenation, which highlights the auxiliary effect of the model structure on feature extraction. In order to avoid the influence of the feature dimension on the performance of long short-term memory (LSTM), a multi-layer fully-connected classifier is used instead to predict the label. Through experimental validation, the performance of our model on the multi-label text classification task shows the current optimal level, which fully proves the superiority of our model. INDEX TERMS Artificial neural network, attention mechanism, deep learning, multi-label classification.

Sequence Generation Network Based on Hierarchical Attention for Multi-Charge Prediction

“…The context vector u w can be seen as a high-level representation of a fixed query "what is the informative word" over the words similarly to that used in memory networks. The word context vector u w is randomly initialized and jointly learned during the training process [27] [28] [29].…”

A Multi-label Classification of Disaster-Related Tweets with Enhanced Word Embedding Ensemble Convolutional Neural Network Model

“…However, more research and inventions are required to improve social media text data classifications. Data collection, preprocessing, feature selection, Construction of a new classification methodology, Training, hyper-parameter fine-tuning, and evaluation of the proposed method are the customary phases of a text classification system [7]. The microblog messages are categorized under eight categories: Disaster kind, Location, Dead and Injured People, Help Request, Infrastructure damage, Search and Rescue, Weather-related information, and Non-relevant information to reach victims of disasters more quickly and efficiently.…”