Multi‐level Deep Correlative Networks for Multi‐modal Sentiment Analysis

Cai, Guoyong; Lyu, Guangrui; Lin, Yang; Wen, Yimin

doi:10.1049/cje.2020.09.003

Cited by 6 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A multilevel deep correlative network for multimodal sentiment analysis is proposed (Cai et al, 2020) Reduce the semantic gap by simultaneously analysing the middle-level semantic features of images and the hierarchical deep correlations.…”

Section: Methods Advantages Disadvantagesmentioning

confidence: 99%

“…Multimodal sentiment analysis is gaining popularity because it expanded traditional SA based on texts to multimodal content, which provide richer effective data. Multilevel deep correlative networks were developed by (Cai et al, 2020). The developed scheme decreases the semantic gap by investigating the deep hierarchical correlations and the middle‐level semantic features.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Effective deep learning based multimodal sentiment analysis from unstructured big data

2022

View full text Add to dashboard Cite

More recently, as images, memes and graphics interchange formats have dominated social feeds, typographic/infographic visual content has emerged as an important social media component. This multimodal text combines text and image, defining a novel visual language that must be analysed because it has the potential to modify, confirm or grade the sentiment's polarity. The problem is how to effectively use information from the visual and textual content in image-text posts. This article presents a new deep learning-based multimodal sentiment analysis (MSA) model using multimodal data such as images, text and multimodal text (image with embedded text). The text analytic unit, the discretization control unit, the picture analytic component and the decision-making component are all included in this system. The discretization unit separates the text from the picture using the variant and channel augmented maximally stable extremal regions (VCA-MSERs) technique, which are then analysed as discrete elements and fed into the appropriate image and text analytics units. The text analytics system utilizes a stacked recurrent neural network with multilevel attention and feedback module (SRNN-MAFM) to detect the sentiment of the text. A deep convolutional neural network (CNN) structure with parallel-dilated convolution and self-attention module (PDC-SAM) is developed to forecast the emotional response to visual content. Finally, the decision component employs a Boolean framework including an OR function to evaluate and classify the output into three fine-grained sentiment classes: positive, neutral and negative. The proposed work is simulated in the python platform using the STS-Gold, Flickr 8k and B-T4SA datasets for sentiment analysis of text and visual and multimodal text. Simulation outcomes proved that the suggested method achieved better accuracy of 97.8%, 97.7% and 90% for text, visual and MSA individually compared to other methods.

show abstract

Section: Methods Advantages Disadvantagesmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Effective deep learning based multimodal sentiment analysis from unstructured big data

2022

View full text Add to dashboard Cite

show abstract

“…This study utilizes a cross modal attention mechanism to fuse features pairs by pairs between different modalities, capturing the correlation between modalities. This stage is called the cross modal feature fusion layer [22]. Then, the obtained pairwise fused feature matrix is concatenated and the internal correlation of modal features is captured through self attention mechanism.…”

Section:  mentioning

confidence: 99%

The Performance of a Temporal Multi-Modal Sentiment Analysis Model Based on Multitask Learning in Social Networks

He,

2024

ijacsa

View full text Add to dashboard Cite

When conducting sentiment analysis on social networks, facing the challenge of temporal and multi-modal data, it is necessary to enable the model to deeply mine and combine information from various modalities. Therefore, this study constructs an emotion analysis model based on multitask learning. This model utilizes a comprehensive framework of convolutional networks, bidirectional gated recurrent units, and multi head selfattention mechanisms to represent single modal temporal features in an innovative way, and adopts a cross modal feature fusion strategy. The experiment showed that the model accomplished 0.83 average precision and a 0.83 F1-value, respectively. In contrast with multi-scale attention (0.69, 0.70), aspect-based sentiment analysis (0.78, 0.74), and long short-term memory network (0.71, 0.78) models, this model demonstrated higher robustness and classification accuracy. Especially in terms of parallel computing efficiency, the acceleration ratio of the model reached 1.61, which is the highest among all compared models, highlighting the potential for time savings in large data volumes. This study has shown good performance in sentiment analysis in social networks, providing a novel perspective for solving complex sentiment classification problems.

show abstract

“…ese metadata can be used to design new recommendation models and improve the recommendation accuracy of the models. e content filtering method [4] makes up for the shortage of collaborative filtering by mining the content features of songs that users have listened to recommend songs that are similar to them. However, content filtering ignores the "synergy law" among users because similar songs have similar properties and similar users have similar preferences.…”

Section: Introductionmentioning

confidence: 99%

Design of a Music Recommendation Model on the Basis of Multilayer Attention Representation

Lü

2022

Scientific Programming

View full text Add to dashboard Cite

Current music recommendation systems can explore the general relationship between the users and songs to recommend music to the users; however, they cannot distinguish the different preferences of different users for the same song. For example, a user may like a song because of the singer, while another user will like it not for the singer but just because of the composition of the song or its melody. A recommender system that knows this difference would be more effective in recommending music to the users. To this end, this paper proposes a music recommendation model based on multilayer attention representation, which learns song representations from multidimensions using user-attribute information and song content information, and mines the preference relationship between users and songs. In order to distinguish the differences in user preferences for multidomain features of songs, a feature-dependent attention network is designed; in order to distinguish the differences in user preferences for different historical behaviors and to explore the temporal dependence of user behaviors, a song-dependent attention network is designed. Finally, the SoftMax function is used to calculate the distribution of users’ preferences for candidate songs and is used to generate recommendations. The experimental results on 30Music and MIGU datasets show that the proposed model achieves significant improvement in recall and MRR compared with the current recommendation models.

show abstract

Multi‐level Deep Correlative Networks for Multi‐modal Sentiment Analysis

Cited by 6 publications

References 25 publications

Effective deep learning based multimodal sentiment analysis from unstructured big data

Effective deep learning based multimodal sentiment analysis from unstructured big data

The Performance of a Temporal Multi-Modal Sentiment Analysis Model Based on Multitask Learning in Social Networks

Design of a Music Recommendation Model on the Basis of Multilayer Attention Representation

Contact Info

Product

Resources

About