Multimodal Sentiment Analysis: Review, Application Domains and Future Directions

Gandhi, Ankita; Adhvaryu, Kinjal; Khanduja, Vidhi

doi:10.1109/punecon52575.2021.9686504

Cited by 9 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent time, multimodal that is using different modalities like audio, visual and textual data processing is gaining much more traction. In study by Gandhi et al (2021), number of multimodal data processing applications are discussed. Classifying hateful memes is a difficult process.…”

Section: Hate Speech Detection In Different Data Modalitiesmentioning

confidence: 99%

Hate speech detection: A comprehensive review of recent works

Gandhi,

Ahir,

Adhvaryu

et al. 2024

Expert Systems

Self Cite

View full text Add to dashboard Cite

There has been surge in the usage of Internet as well as social media platforms which has led to rise in online hate speech targeted on individual or group. In the recent years, hate speech has resulted in one of the challenging problems that can unfurl at a fast pace on digital platforms leading to various issues such as prejudice, violence and even genocide. Considering the acceptance of Artificial Intelligence (AI) and Natural Language Processing (NLP) techniques in varied application domains, it would be intriguing to consider these techniques for automated hate speech detection. In literature, there have been efforts to recognize and categorize hate speech using varied Machine Learning (ML) and Deep Learning (DL) techniques. Hence, considering the need and provocations for hate speech detection we aim to present a comprehensive review that discusses fundamental taxonomy as well as recent advances in the field of online hate speech identification. There is a significant amount of literature related to the initial phases of hate speech detection. The background section provides a detailed explanation of the previous research. The subsequent section that follows is dedicated to examining the recent literature published from the year 2020 onwards. The paper presents some of the hate speech datasets considered for hate speech detection. Furthermore, the paper discusses different data modalities, namely, textual hate speech detection, multi‐modal hate speech detection and multilingual hate speech detection. Apart from systematic review on hate speech detection, the paper also implement several multi‐label models to compare the performance of hate speech detection by employing classic ML technique namely, Logistic Regression and DL technique namely, Long Short‐Term Memory (LSTM) and a multiclass multi‐label architecture. In the implemented architecture, we have derived two new elements to quantify the hatefulness and intensity of hatred to improve the results for hate speech detection using Indonesian tweet dataset. Empirical Analysis of the model reveals that the implemented approach outperforms and is able to achieve improved results for the underlying dataset.

show abstract

Section: Hate Speech Detection In Different Data Modalitiesmentioning

confidence: 99%

Hate speech detection: A comprehensive review of recent works

Gandhi,

Ahir,

Adhvaryu

et al. 2024

Expert Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…SA typically involves the development of a model which classifies opinion into labeled polarities such as positive, negative, and or neutral classes [1]. These varied polarities necessitate that the raw data availabilities be utilized for mining opinions while also identifying their sentiments, as prior literature focuses on textual data, which may fail to generate accurate results [2]. An ideal source of such multimodal information is a video, which provides visual frames and information on spoken language's acoustic and textual representation [2,4].…”

mentioning

confidence: 99%

“…These varied polarities necessitate that the raw data availabilities be utilized for mining opinions while also identifying their sentiments, as prior literature focuses on textual data, which may fail to generate accurate results [2]. An ideal source of such multimodal information is a video, which provides visual frames and information on spoken language's acoustic and textual representation [2,4]. The integration of these varied data is referred to as Multimodal Sentiment Analysis (MSA).…”

mentioning

confidence: 99%

Fuzzy Layered Convolution Neutral Network for Feature Level Fusion Based On Multimodal Sentiment Classification

Ayodele¹,

Harun²,

Yusoff³

2023

EmAIT

View full text Add to dashboard Cite

Multimodal sentiment analysis (MSA) is one of the core research topics of natural language processing (NLP). MSA has become a challenge for scholars and is equally complicated for an appliance to comprehend. One study that supports MSdifficulties is the MSA, which is learning opinions, emotions, and attitudes in an audio-visual format. In order words, using such diverse modalities to obtain opinions and identify emotions is necessary. Such utilization can be achieved via modality datafusion;such as feature fusion. In handling the data fusion of such diverse modalities while obtaining high performance, a typical machine learning algorithm is Deep Learning (DL), particularly the Convolutional Neutral Network (CNN), which has the capacity to handle tasks of great intricacy and difficulty. In this paper, we present a CNN architecture with an integrated layer via fuzzy methodologies for MSA, a task yet to be explored in improving the accuracy performance of CNN for diverse inputs. Experiments conducted on a benchmark multimodal dataset, MOSI, obtaining 37.5% and 81% on seven (7) class and binary classification respectively, reveals an improved accuracy performance compared with the typical CNN, which acquired 28.9% and 78%, respectively.

show abstract