Self-Attentive Feature-Level Fusion for Multimodal Emotion Detection

Hazarika, Devamanyu; Gorantla, Sruthi; Poria, Soujanya; Zimmermann, Roger

doi:10.1109/mipr.2018.00043

Cited by 47 publications

(21 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although this variation reduces the total parameter sizes of the network, it still does not benefit the model and gives a poorer performance to simple concatenation. Numerous other fusion methods such as tensor fusion (Zadeh et al, 2017), compact bilinear pooling (Gao et al, 2016), attention-based fusion (Poria et al, 2017;Hazarika et al, 2018), etc. are applicable, whose analyses, however, is not the focus of this paper.…”

Section: Resultsmentioning

confidence: 99%

Modeling Inter-Aspect Dependencies for Aspect-Based Sentiment Analysis

Hazarika¹,

Poria²,

Vij³

et al. 2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

Self Cite

View full text Add to dashboard Cite

Aspect-based Sentiment Analysis is a finegrained task of sentiment classification for multiple aspects in a sentence. Present neuralbased models exploit aspect and its contextual information in the sentence but largely ignore the inter-aspect dependencies. In this paper, we incorporate this pattern by simultaneous classification of all aspects in a sentence along with temporal dependency processing of their corresponding sentence representations using recurrent networks. Results on the benchmark SemEval 2014 dataset suggest the effectiveness of our proposed approach.

show abstract

Section: Resultsmentioning

confidence: 99%

Modeling Inter-Aspect Dependencies for Aspect-Based Sentiment Analysis

Hazarika¹,

Poria²,

Vij³

et al. 2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: Hu

Self Cite

View full text Add to dashboard Cite

show abstract

“…Using deep learning to classify facial expressions is usually learning how to use strong supervision methods [15] Barros et al proposed a network model based on the topological structure of VGG-16 for the formalization of the Facial Channel neural network for Facial Expression Recognition (FER) [24]. Koujan et al proposed a CNN that recognized human emotions from a single face image [25].…”

Section: Related Workmentioning

confidence: 99%

A Lightweight Convolutional Neural Network for Real-Time Facial Expression Detection

2021

View full text Add to dashboard Cite

“…MED means to perform emotion detection from a multimodal learning perspective [3], [26]. A large amount of current research, such as [27]- [31], have been dependent on multimodal techniques for emotion detection. These exhibit good performances in emotion detection systems [32]- [34], thus promoting the use of multimodality.…”

Section: Related Workmentioning

confidence: 99%

Different Contextual Window Sizes Based RNNs for Multimodal Emotion Detection in Interactive Conversations

Lai

Chen

2020

IEEE Access

View full text Add to dashboard Cite

Multimodal emotion detection (MED) in interactive conversations is extremely important for improving the overall human-computer interaction experience. Present research methods in this domain do not explicitly distinguish the contexts of a test utterance in a meaningful way while classifying emotions in conversations. In this paper, we propose a model, named different contextual window sizes based recurrent neural networks (DCWS-RNNs), to differentiate the contexts. The model has four recurrent neural networks (RNNs) that use different contextual window sizes. These window sizes can represent the implicit weights of different aspects of contexts. Further, four RNNs are independently to model the different aspects of contexts into memories. Such memories with the test utterance are then merged using attention-based multiple hops. Experiments show DCWS-RNNs outperforms the compared methods on both the IEMOCAP and AVEC datasets. Case studies on the IEMOCAP dataset also demonstrate that our model has excellent performance to capture the emotional dependent utterance that is most relevant to the test utterance and assigned to the highest attention score. INDEX TERMS Interactive conversations, contextual window sizes, emotion detection, multimodal, recurrent neural network.

show abstract

Self-Attentive Feature-Level Fusion for Multimodal Emotion Detection

Cited by 47 publications

References 18 publications

Modeling Inter-Aspect Dependencies for Aspect-Based Sentiment Analysis

Modeling Inter-Aspect Dependencies for Aspect-Based Sentiment Analysis

A Lightweight Convolutional Neural Network for Real-Time Facial Expression Detection

Different Contextual Window Sizes Based RNNs for Multimodal Emotion Detection in Interactive Conversations

Contact Info

Product

Resources

About