Sentiment and Context-Aware Hybrid DNN With Attention for Text Sentiment Classification

Abstract

A massive volume of unstructured data in the form of comments, opinions, and other sorts of data is generated in real-time with the growth of web 2.0. Due to the unstructured nature of the data, building an accurate predictive model for sentiment analysis remains a challenging task. While various DNN architectures have been applied to sentiment analysis with encouraging results, they suffer from high dimensional feature space and consider various features equally. State-of-the-art methods cannot properly leverage semantic and sentiment knowledge to extract meaningful relevant contextual sentiment features.This paper proposes a sentiment and context-aware hybrid DNN model with an attention mechanism that {intelligently learns and highlights salient features of relevant sentiment context in the text. We first use integrated wide coverage sentiment lexicons to identify text sentiment features then leverage bidirectional encoder representation from transformers to produce sentiment-enhanced word embeddings for text semantic extraction. After that, the proposed approach adapts the BiLSTM to capture both word order/contextual text semantic information and the long-dependency relation in the word sequence. Our model also employs an attention mechanism to assign weight to features and give greater significance to salient features in the word sequence. Finally, CNN is utilized to reduce the dimensionality of feature space and extract the local key features for sentiment analysis. The effectiveness of the proposed model is evaluated on real-world benchmark datasets demonstrating that the proposed model significantly improves the accuracy of existing text sentiment classification.

An automated learning model for twitter sentiment analysis using Ranger AdaBelief optimizer based Bidirectional Long Short Term Memory

“…However, the word representation present in the framework affects the classification accuracy of the whole framework due to the dimensionality of the word that creates impact in ability of the model. Khan et al (2023) have introduced a Sentiment and Context Aware Attention-based Hybrid Deep Neural Network (SCA-HDNN) model along with attention mechanism which effectively figure out the salient features in the text. Initially, integrated wide coverage sentiment lexicons are used to detect the sentiment features and BERT model is used to detect the sentiment of the extracted words.…”

A novel multi-layer feature fusion-based BERT-CNN for sentence representation learning and classification

“…The authors (Liu et al, 2023a(Liu et al, , 2023b proposed a new variant of BERT-BASE, which uses explicitly defined word embedding extracted from the sentences, and on the SST-2 data set, it achieved an accuracy of 93.85%. The authors (Khan et al, 2023) proposed a hybrid model by adding an attention mechanism to BERT-CNN and achieved an accuracy of 91.90% on the SST-2 data set. Athira et al (2023) exploited BioBERT, a BERT model developed for biomedical language representation and tested on the QQP data set.…”

Fake news stance detection using selective features and FakeNET

“…Different types of deep learning models [20][21][22][23] have gained a lot of popularity in NLP tasks such as question answering [24,25], finding the semantic similarity in texts [26,27], text analysis [28,29], etc. The semantic similarity of the two questions is calculated using Siamese MaLSTM in [30].…”