A New Hybrid Based on Long Short-Term Memory Network with Spotted Hyena Optimization Algorithm for Multi-Label Text Classification

Abstract

An essential work in natural language processing is the Multi-Label Text Classification (MLTC). The purpose of the MLTC is to assign multiple labels to each document. Traditional text classification methods, such as machine learning usually involve data scattering and failure to discover relationships between data. With the development of deep learning algorithms, many authors have used deep learning in MLTC. In this paper, a novel model called Spotted Hyena Optimizer (SHO)-Long Short-Term Memory (SHO-LSTM) for MLTC based on LSTM network and SHO algorithm is proposed. In the LSTM network, the Skip-gram method is used to embed words into the vector space. The new model uses the SHO algorithm to optimize the initial weight of the LSTM network. Adjusting the weight matrix in LSTM is a major challenge. If the weight of the neurons to be accurate, then the accuracy of the output will be higher. The SHO algorithm is a population-based meta-heuristic algorithm that works based on the mass hunting behavior of spotted hyenas. In this algorithm, each solution of the problem is coded as a hyena. Then the hyenas are approached to the optimal answer by following the hyena of the leader. Four datasets are used (RCV1-v2, EUR-Lex, Reuters-21578, and Bookmarks) to evaluate the proposed model. The assessments demonstrate that the proposed model has a higher accuracy rate than LSTM, Genetic Algorithm-LSTM (GA-LSTM), Particle Swarm Optimization-LSTM (PSO-LSTM), Artificial Bee Colony-LSTM (ABC-LSTM), Harmony Algorithm Search-LSTM (HAS-LSTM), and Differential Evolution-LSTM (DE-LSTM). The improvement of SHO-LSTM model accuracy for four datasets compared to LSTM is 7.52%, 7.12%, 1.92%, and 4.90%, respectively.

SHO-CNN: A Metaheuristic Optimization of a Convolutional Neural Network for Multi-Label News Classification

“…Furthermore, SHO-CNN also achieves higher performance when compared to the baseline and other state-of-the-art approaches. LSTM, an upgraded version of CNN, is optimized using SHO [45], resulting in a higher hamming loss than SHO-CNN but superior micro-f1 and macro-f1 performance. Table 6 shows a comparison of the proposed SHO-CNN model with various state-of-the-art approaches on the RCV1-v2, Reuters21578, Slashdot, and NELA-GT-2019 datasets.…”

“…Table 6 shows a comparison of the proposed SHO-CNN model with various state-of-the-art approaches on the RCV1-v2, Reuters21578, Slashdot, and NELA-GT-2019 datasets. [92] 0.872 0.855 0.682 LP [93] 0.874 0.858 0.691 CNN [25] 0.891 0.849 0.727 BERT [94] 0.733 0.877 0.667 CNN-RNN [95] 0.872 0.849 0.752 SGM [96] 0.814 0.869 0.712 SGM-GE [97] 0.753 0.878 0.762 Seq2Set [98] 0.736 0.879 0.751 ML-R [50] 0.793 0.898 0.695 Seq2Tree [99] 0.732 0.868 0.700 TextRCNN [100] 0.765 0.815 0.592 HiAGM [100] 0.768 0.839 0.633 HTCInfoMax [101] 0.748 0.835 0.627 HiMatch [102] 0.754 0.847 0.641 SHO-LSTM [45] 0.737 0.913 0.781 SHO-CNN (Proposed) 0.722 0.906 0.776…”

“…Reuters21578 BR [91] 0.434 0.835 0.402 PCC [103] 0.442 0.834 0.398 seq2seq-RNN [104] 0.384 0.858 0.437 BERT [94] 0.261 0.905 0.640 LBA [105] 0.254 0.908 0.675 ML-KNN [106] 0.376 0.732 0.253 ML-ARAM [61] 0.363 0.626 0.163 ML-RNN [104] 0.332 0.858 0.457 LaMP [107] 0.291 0.889 0.560 MPVAE [108] 0.267 0.887 0.542 MrMP [109] 0.254 0.893 0.591 Class-wise cluster + FS [110] -0.800 0.725 CB-NTR [111] 0.247 0.907 0.633 SHO-LSTM [45] 0.240 0.910 0.690 SHO-CNN (Proposed) 0.237 0.908 0.665 Slashdot BR [91] 0.052 0.486 0.362 BR-Support [112] 0.055 0.516 0.357 PCC [103] 0.056 0.480 0.279 seq2seq-RNN [104] 0.058 0.528 0.270 set-RNN [112] 0.053 0.538 0.310 BERT [94] 0.037 0.583 0.380 LBA [105] 0.038 0.582 0.410 CC [92] 0.057 0.480 0.373 CNN [25] 0.049 0.512 0.412 CNN-RNN [95] 0.046 0.530 0.469 MAGNET [112] 0.039 0.568 0.475 SHO-LSTM [45] 0.038 0.650 0.492 SHO-CNN (Proposed) 0.037 0.632 0.453 NELA-GT-2019 BR [91] 0.033 0.870 0.792 BR-Support [112] 0.040 0.846 0.783 PCC [103] 0.009 0.922 0.913 LBA [105] 0.008 0.953 0.9420 CNN [25] 0.008 0.934 0.926 CNN-RNN [95] 0.008 0.935 0.944 SHO-LSTM [45] 0.007 0.972 0.970 SHO-CNN (Proposed) 0.006 0.963 0.956…”

“…These characteristics of SHO make it highly suitable for exploring new domains [40,41]. SHO has already achieved global success in optimization and is regarded as one of the top exploration techniques [42][43][44][45]. Therefore, the purpose of this study is to utilize the highest strengths of metaheuristic techniques to overcome CNN's limitations in the field of news MLTC for improved classification accuracy and efficiency.…”

A deep learning framework for layer-wise porosity prediction in metal powder bed fusion using thermal signatures

“…Deep learning has become the AI/ML technique of choice in the last few years, due to its groundbreaking success in numerous real-world applications (Aghazadeh & Gharehchopogh, 2018;Asghari et al, 2021;Collobert & Weston, 2008;Khataei Maragheh et al, 2022;Krizhevsky et al, 2017;Lin et al, 2019;Niu et al, 2019). Four machine learning methods were adopted by Paulson et al (2020) to model the relationships between thermal histories and surface porosity formation in Ti6Al4V during single line scans using L-PBF.…”