Modeling Incongruity between Modalities for Multimodal Sarcasm Detection

Wu, Yang; Zhao, Yanyan; Lu, Xin; Qin, Bing; Wu, Yung-Hsien; Sheng, Jian; Li, Jinlong

doi:10.1109/mmul.2021.3069097

Cited by 32 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experimental results revealed that the constructed model was more applicable in contrast to others. An IWAN (incongruity-aware attention network) model was devised by Wu et al for detecting the sarcasm in which word-level incongruity was considered among modalities through a scoring approach [ 28 ]. This approach was capable of assigning the larger weights to words with incongruent modalities.…”

Section: Literature Reviewmentioning

confidence: 99%

Ensemble Classification Approach for Sarcasm Detection

Godara

Batra

Aron

et al. 2021

Behavioural Neurology

View full text Add to dashboard Cite

Cognitive science is a technology which focuses on analyzing the human brain using the application of DM. The databases are utilized to gather and store the large volume of data. The authenticated information is extracted using measures. This research work is based on detecting the sarcasm from the text data. This research work introduces a scheme to detect sarcasm based on PCA algorithm, K -means algorithm, and ensemble classification. The four ensemble classifiers are designed with the objective of detecting the sarcasm. The first ensemble classification algorithm (SKD) is the combination of SVM, KNN, and decision tree. In the second ensemble classifier (SLD), SVM, logistic regression, and decision tree classifiers are combined for the sarcasm detection. In the third ensemble model (MLD), MLP, logistic regression, and decision tree are combined, and the last one (SLM) is the combination of MLP, logistic regression, and SVM. The proposed model is implemented in Python and tested on five datasets of different sizes. The performance of the models is tested with regard to various metrics.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Ensemble Classification Approach for Sarcasm Detection

Godara

Batra

Aron

et al. 2021

Behavioural Neurology

View full text Add to dashboard Cite

show abstract

“…Secondly, for the text feature sequence ( S l ) in linguistic mode, most linguistic features are embedded through Glove [ 37 ]. However, in recent studies [ 38 ], such as the advanced ICCN [ 39 ] model, the pretraining BERT model is used as the feature extractor of text discourse.…”

Section: The Proposed Methodsmentioning

confidence: 99%

Dynamic Invariant-Specific Representation Fusion Network for Multimodal Sentiment Analysis

Yanga

Zhang

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Multimodal sentiment analysis (MSA) aims to infer emotions from linguistic, auditory, and visual sequences. Multimodal information representation method and fusion technology are keys to MSA. However, the problem of difficulty in fully obtaining heterogeneous data interactions in MSA usually exists. To solve these problems, a new framework, namely, dynamic invariant-specific representation fusion network (DISRFN), is put forward in this study. Firstly, in order to effectively utilize redundant information, the joint domain separation representations of all modes are obtained through the improved joint domain separation network. Then, the hierarchical graph fusion net (HGFN) is used for dynamically fusing each representation to obtain the interaction of multimodal data for guidance in the sentiment analysis. Moreover, comparative experiments are performed on popular MSA data sets MOSI and MOSEI, and the research on fusion strategy, loss function ablation, and similarity loss function analysis experiments is designed. The experimental results verify the effectiveness of the DISRFN framework and loss function.

show abstract

“…Incongruity aware attention network (IWAN) [13] was proposed that combines multimodal data such as textual, visual, and audio information and uses the incongruity between the features to detect sarcasm. To extract characteristics from various data modes, various models were utilised.…”

Section: Related Workmentioning

confidence: 99%

Explaining Sarcasm of Tweets using Attention Mechanism

Keerthana,

Singh,

Saini

et al. 2023

SCPE

View full text Add to dashboard Cite

Emotion identification from text can help boost the effectiveness of sentiment analysis models. Sarcasm is one of the more difficult emotions to detect, particularly in textual data. Even though several models for detecting sarcasm have been presented, their performance falls way short of that of other emotion detection models. As a result, few strategies have been introduced in the paper that helped to enhance the performance of sarcasm detection models. To compare performance, the model was tested using the TweetEval benchmark dataset. On the TweetEval benchmark, the technique proposed in this paper has established a new state-of-the-art. Besides the low performance, interpretability of existing sarcasm detection models are lacking compared to other emotion detection models like hate speech and anger. Therefore, an attention-based interpretability technique has been proposed in this paper that interprets the token importance for a certain decision of sarcasm detection model. The results of the interpretability technique aid in our comprehension of the contextual embeddings of the input tokens that the model has paid the greatest attention to while making a particular decision which outperforms existing transformer-based interpretability techniques, particularly in terms of visualisations.

show abstract

Modeling Incongruity between Modalities for Multimodal Sarcasm Detection

Cited by 32 publications

References 14 publications

Ensemble Classification Approach for Sarcasm Detection

Ensemble Classification Approach for Sarcasm Detection

Dynamic Invariant-Specific Representation Fusion Network for Multimodal Sentiment Analysis

Explaining Sarcasm of Tweets using Attention Mechanism

Contact Info

Product

Resources

About