History-based attention in Seq2Seq model for multi-label text classification

Xiao, Yaoqiang; Li, Yi; Yuan, Jing; Guo, Songrui; Xiao, Yi; Li, Zhiyong

doi:10.1016/j.knosys.2021.107094

Cited by 37 publications

(11 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning methods have shown great potential for acquiring language and image analysis from text libraries and text vector within neural networks, and are widely used to analyze audio and video, according to [ 12 ]. In [ 13 , 14 ], CNN is prominent among various types of neural networks and is a vital technology enabling many deep learning applications.” As shown in [ 15 , 38 ], CNN has effectively categorized news data, assessed attitudes in tweets and movie reviews, and performed other tasks. The author suggests a multilabel technique built on layered aggregation using several methods in [ 16 ].…”

Section: Literature Reviewmentioning

confidence: 99%

“…When it comes to determining which label collection belongs to a certain research instance, neural models are essential. Moreover, [ 38 ] introduces a Neural Network (NN) model as an additional machine learning technique. To improve the NN’s applicability for machine classification problems, modifications to a fluid, adaptive resonance map are proposed.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Enhancing aspect-based multi-labeling with ensemble learning for ethical logistics

Almazroi,

Ayub

2024

PLoS ONE

View full text Add to dashboard Cite

In the dynamic domain of logistics, effective communication is essential for streamlined operations. Our innovative solution, the Multi-Labeling Ensemble (MLEn), tackles the intricate task of extracting multi-labeled data, employing advanced techniques for accurate preprocessing of textual data through the NLTK toolkit. This approach is carefully tailored to the prevailing language used in logistics communication. MLEn utilizes innovative methods, including sentiment intensity analysis, Word2Vec, and Doc2Vec, ensuring comprehensive feature extraction. This proves particularly suitable for logistics in e-commerce, capturing nuanced communication essential for efficient operations. Ethical considerations are a cornerstone in logistics communication, and MLEn plays a pivotal role in detecting and categorizing inappropriate language, aligning inherently with ethical norms. Leveraging Tf-IDF and Vader for feature enhancement, MLEn adeptly discerns and labels ethically sensitive content in logistics communication. Across diverse datasets, including Emotions, MLEn consistently achieves impressive accuracy levels ranging from 92% to 97%, establishing its superiority in the logistics context. Particularly, our proposed method, DenseNet-EHO, outperforms BERT by 8% and surpasses other techniques by a 15-25% efficiency. A comprehensive analysis, considering metrics such as precision, recall, F1-score, Ranking Loss, Jaccard Similarity, AUC-ROC, sensitivity, and time complexity, underscores DenseNet-EHO’s efficiency, aligning with the practical demands within the logistics track. Our research significantly contributes to enhancing precision, diversity, and computational efficiency in aspect-based sentiment analysis within logistics. By integrating cutting-edge preprocessing, sentiment intensity analysis, and vectorization, MLEn emerges as a robust framework for multi-label datasets, consistently outperforming conventional approaches and giving outstanding precision, accuracy, and efficiency in the logistics field.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Enhancing aspect-based multi-labeling with ensemble learning for ethical logistics

Almazroi,

Ayub

2024

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…In recent years, transformer-style models achieved many good results in various tasks. Subsequently, attention mechanisms have become more common and are widely used in classifcation tasks, such as sentiment classifcation [29], musical instrument recognition [30], visual recommender systems [31], multilabel text classifcation [32], and multiple protein subcellular location prediction [33].…”

Section: Attention Mechanismmentioning

confidence: 99%

Improved Double‐Layer Structure Multilabel Classification Model via Optimal Sequence and Attention Mechanism

Liu

Tan

2022

Complexity

View full text Add to dashboard Cite

Multilabel classification is a key research topic in the machine learning field. In this study, the author put forward a two/two-layer chain classification algorithm with optimal sequence based on the attention mechanism. This algorithm is a classification model with a two-layer structure. By introducing an attention mechanism, this study analyzes the key attributes to achieve the goal of classification. To solve the problem of algorithm accuracy degradation caused by the order of classifiers, we adopt the OSS (optimal sequence selection) algorithm to find the optimal sequence of tags. The test results based on the actual dataset show that the ATDCC-OS algorithm has good performance on all performance evaluation metrics. The average accuracy of this algorithm is over 80%. The microaverage AUC performance reaches 0.96. In terms of coverage performance, its coverage performance is below 10%. The comprehensive result of single error performance is the best. The loss performance is about 0.03. The purpose of the ATDCC-OS algorithm proposed in the study is to help improve the accuracy of multilabel classification so as to obtain more effective data information.

show abstract

“…Multilabel classification problems are common in various fields, such as disease diagnosis, where a patient can have multiple conditions like heart disease and diabetes. Multilabel classification has been widely used in text recognition (Xiao et al , 2021), video classification (Karagoz et al , 2021), clinical detection and other fields (Blanco et al , 2020).…”

Section: Related Background Theorymentioning

confidence: 99%

The multilabel fault diagnosis model of bearing based on integrated convolutional neural network and gated recurrent unit

Han

Zhang

et al. 2021

IJICC

View full text Add to dashboard Cite

PurposeIntelligent diagnosis of equipment faults can effectively avoid the shutdown caused by equipment faults and improve the safety of the equipment. At present, the diagnosis of various kinds of bearing fault information, such as the occurrence, location and degree of fault, can be carried out by machine learning and deep learning and realized through the multiclassification method. However, the multiclassification method is not perfect in distinguishing similar fault categories and visual representation of fault information. To improve the above shortcomings, an end-to-end fault multilabel classification model is proposed for bearing fault diagnosis.Design/methodology/approachIn this model, the labels of each bearing are binarized by using the binary relevance method. Then, the integrated convolutional neural network and gated recurrent unit (CNN-GRU) is employed to classify faults. Different from the general CNN networks, the CNN-GRU network adds multiple GRU layers after the convolutional layers and the pool layers.FindingsThe Paderborn University bearing dataset is utilized to demonstrate the practicability of the model. The experimental results show that the average accuracy in test set is 99.7%, and the proposed network is better than multilayer perceptron and CNN in fault diagnosis of bearing, and the multilabel classification method is superior to the multiclassification method. Consequently, the model can intuitively classify faults with higher accuracy.Originality/valueThe fault labels of each bearing are labeled according to the failure or not, the fault location, the damage mode and the damage degree, and then the binary value is obtained. The multilabel problem is transformed into a binary classification problem of each fault label by the binary relevance method, and the predicted probability value of each fault label is directly output in the output layer, which visually distinguishes different fault conditions.

show abstract

History-based attention in Seq2Seq model for multi-label text classification

Cited by 37 publications

References 57 publications

Enhancing aspect-based multi-labeling with ensemble learning for ethical logistics

Enhancing aspect-based multi-labeling with ensemble learning for ethical logistics

Improved Double‐Layer Structure Multilabel Classification Model via Optimal Sequence and Attention Mechanism

The multilabel fault diagnosis model of bearing based on integrated convolutional neural network and gated recurrent unit

Contact Info

Product

Resources

About