Multimodal Sentiment Analysis: A Survey of Methods, Trends, and Challenges

Abstract

Sentiment analysis has come long way since it was introduced as a natural language processing task nearly 20 years ago. Sentiment analysis aims to extract the underlying attitudes and opinions toward an entity. It has become a powerful tool used by governments, businesses, medicine, marketing etc. The traditional sentiment analysis model focuses mainly on text content. However, technological advances have allowed people to express their opinions and feelings through audio, image and video channels. As a result, sentiment analysis is shifting from unimodality to multimodality. Multimodal sentiment analysis brings new opportunities with the rapid increase of sentiment analysis as complementary data streams enable improved and deeper sentiment detection which goes beyond text-based analysis. Audio and video channels are included in multimodal sentiment analysis in terms of broadness. People have been working on different approaches to improve sentiment analysis system performance by employing complex deep neural architectures. Recently, sentiment analysis has achieved significant success using the transformer-based model. This paper presents a comprehensive study of different sentiment analysis approaches, applications, challenges and resources then concludes that it holds tremendous potential. The primary motivation of this survey is to highlight changing trends in the unimodality to multimodality for solving sentiment analysis tasks.

Design and Efficacy of a Data Lake Architecture for Multimodal Emotion Feature Extraction in Social Media

“…A comprehensive exploration into sentiment analysis (SA) and emotion detection (ED) unfolds in this scholarly review, emphasizing the crucial role of recurrent neural networks and their architectural variants, in handling textual, visual, and multimodal inputs on social networking platforms [25]. The comprehensive surveys navigate the evolution of SA from text-based models to the realm of multimodality, exploring diverse approaches, applications, challenges, and the transformative impact of deep neural architectures, and reflecting the dynamic shift in SA trends [26,27]. Presenting an innovative multimodal SA system, the study [28] meticulously integrates text-and image-based components, utilizing deep learning for feature learning and classification.…”

Design of Emotion Analysis Model IABC-Deep Learning-based for Vocal Performance

“…These models are first pre trained on large-scale corpora and then fine-tuned on specific tasks. Pre trained models can capture rich language knowledge, significantly improving the performance of various NLP tasks, including sentiment analysis [16]. Meanwhile, knowledge graphs are also a structured way of representing knowledge, which can be used to represent entities and their relationships.…”

Multimodal Audio Emotion Recognition with Graph-based Consensus Pseudolabeling

“…Furthermore, [Bagadi 2021, Deng et al 2021] have explored the incorporation of textual information in audio emotion recognition tasks. Text can provide complementary information, such as the explicit expression of emotions and the topics discussed [Das and Singh 2023]. Automatic text extraction methods, such as automatic speech recognition systems, can transcribe the spoken content and provide textual representations of the audio.…”