An Improved Model for Analyzing Textual Sentiment Based on a Deep Neural Network Using Multi-Head Attention Mechanism

Abstract

Due to the increasing growth of social media content on websites such as Twitter and Facebook, analyzing textual sentiment has become a challenging task. Therefore, many studies have focused on textual sentiment analysis. Recently, deep learning models, such as convolutional neural networks and long short-term memory, have achieved promising performance in sentiment analysis. These models have proven their ability to cope with the arbitrary length of sequences. However, when they are used in the feature extraction layer, the feature distance is highly dimensional, the text data are sparse, and they assign equal importance to various features. To address these issues, we propose a hybrid model that combines a deep neural network with a multi-head attention mechanism (DNN–MHAT). In the DNN–MHAT model, we first design an improved deep neural network to capture the text’s actual context and extract the local features of position invariants by combining recurrent bidirectional long short-term memory units (Bi-LSTM) with a convolutional neural network (CNN). Second, we present a multi-head attention mechanism to capture the words in the text that are significantly related to long space and encoding dependencies, which adds a different focus to the information outputted from the hidden layers of BiLSTM. Finally, a global average pooling is applied for transforming the vector into a high-level sentiment representation to avoid model overfitting, and a sigmoid classifier is applied to carry out the sentiment polarity classification of texts. The DNN–MHAT model is tested on four reviews and two Twitter datasets. The results of the experiments illustrate the effectiveness of the DNN–MHAT model, which achieved excellent performance compared to the state-of-the-art baseline methods based on short tweets and long reviews.

Research on deep reinforcement learning basketball robot shooting skills improvement based on end to end architecture and multi-modal perception

“…The attention mechanism is a machine learning technique that enables the model to focus on the most relevant and important parts when processing input data, thereby improving the performance and efficiency of the model (Sharaf Al-deen et al, 2021 ). The attention mechanism is inspired by human visual attention, that is, when humans observe a scene, they will automatically focus on the region of interest and ignore other irrelevant regions.…”

BiLSTM Model With Attention Mechanism for Sentiment Classification on Chinese Mixed Text Comments

“…The summaries of the literature review can be shown in TABLE I. CNN Ref [7] Stacked residual LSTM Ref [9] SR-LSTM Ref [10] CNN-LSTM Ref [11] Co-LSTM Ref [12] CNN-RNN-LSTM Ref [13] Self-Attention Ref [14] CNN-BiLSTM model with a scalable multi-channel dilated joint design Ref [16] DNN-Multi-Head Attention Ref [17] pre-Attention based BiLSTM Ref [19] BERT Ref [20] multi-modal recurrent neural network-based Attention Ref [21] Attention over Attention…”

Enhancing decision-making support by mining social media data with social network analysis

“…The authors’ model addresses the complexities of other languages with a character embedding. Al-deen et al ( 2021 ) proposed a combination of deep neural networks with a multi-head attention mechanism. They also conducted sentiment analysis and text classification tasks using a large Twitter dataset on the major US airline problems to experiment the performance of their model.…”