Multi-level context extraction and attention-based contextual inter-modal fusion for multimodal sentiment analysis and emotion classification

Huddar, Mahesh G.; Sannakki, Sanjeev S.; Rajpurohit, Vijay S.

doi:10.1007/s13735-019-00185-8

Cited by 29 publications

(10 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…e canonical correlation coefficient measures the strength of the connection between these two sets of variables. is maximization technique aims to map the high-dimensional relationship between the two sets of variables to some typical variables [17].…”

Section: Research Methods Of Neuronal Apoptosismentioning

confidence: 99%

Neuronal Apoptosis in Patients with Liver Cirrhosis and Neuronal Epileptiform Discharge Model Based upon Multi-Modal Fusion Deep Learning

Chi

Wang

et al. 2022

Journal of Healthcare Engineering

View full text Add to dashboard Cite

Neurons refer to nerve cells. Each neuron is connected with thousands of other neurons to form a corresponding functional area and carry out complex communication with other functional areas. Its importance to the human body is self-evident. There are also many scholars studying the mechanism of apoptosis. This paper proposes a study of neuronal apoptosis in patients with liver cirrhosis and neuronal epileptiform discharge models based on multi-modal fusion deep learning, aiming to study the influencing factors of abnormal neuronal discharge in the brain. The method in this paper is to study multi-modal information fusion methods, perform Bayesian inference, and analyze multi-modal medical data. The function of these research methods is to obtain the relationship between the independence of information and the intersection of information among modalities. In the neuronal epileptiform discharge model, the mRNA expression level of the necroptotic signaling pathway related protein was detected, and the mechanism of neuronal necrosis in patients with liver cirrhosis was explored. Experiments show that the neuron recognition rate has been increased from 67.2% to 84.5%, and the time has been reduced, proving the effectiveness of deep learning.

show abstract

Section: Research Methods Of Neuronal Apoptosismentioning

confidence: 99%

Neuronal Apoptosis in Patients with Liver Cirrhosis and Neuronal Epileptiform Discharge Model Based upon Multi-Modal Fusion Deep Learning

Chi

Wang

et al. 2022

Journal of Healthcare Engineering

View full text Add to dashboard Cite

show abstract

“…Different from traditional manual retrieval methods, automatic retrieval will save a lot of labor costs. At the same time, compared with manual analysis, improving the accuracy of analysis will be a difficult problem for automatic analysis [12]. As an important means of automatic music retrieval, classifying music according to the expressed emotion is attracting the attention of researchers from different fields.…”

Section: Introductionmentioning

confidence: 99%

[Retracted] Music Emotion Research Based on Reinforcement Learning and Multimodal Information

2022

Journal of Mathematics

View full text Add to dashboard Cite

Music is an important carrier of emotion and an indispensable factor in people’s daily life. With the rapid growth of digital music, people’s demand for music emotion analysis and retrieval is also increasing. With the rapid development of Internet technology, digital music has been derived continuously, and automatic recognition of music emotion has become the main research focus. For music, emotion is the most essential feature and the deepest inner feeling. Under the ubiquitous information environment, revealing the deep semantic information of multimodal information resources and providing users with integrated information services has important research and application value. In this paper, a multimodal fusion algorithm for music emotion analysis is proposed, and a dynamic model based on reinforcement learning is constructed to improve the analysis accuracy. The model dynamically adjusts the emotional analysis results by learning the user’s behavior, so as to realize the personalized customization of the user’s emotional preference.

show abstract

“…M.G. Huddar, et al, [13] used multimodal corpus of sentiment intensity dataset and interactive motional dyadic motion capture dataset for emotion detection and multimodal sentiment analysis. Initially, Z-score standardization approach was employed on the audio modality for voice intensity threshold and voice normalization, and then 6392 feature vectors were extracted from the audio signals: arithmetic mean, pitch, standard deviation, amplitude mean, voice intensity, root quadratic mean, etc.…”

Section: Related Workmentioning

confidence: 99%

MALO-LSTM: Multimodal Sentiment Analysis Using Modified Ant Lion Optimization with Long Short Term Memory Network

2022

IJIES

View full text Add to dashboard Cite

In recent times, multimodal sentiment analysis is the most researched topic, due to the availability of huge amount of multimodal content. Generally, multimodal sentiment analysis uses text, audio and visual representations for effective sentiment recognition. The detection of sentiment in the natural language is a tricky process even for humans, so making it automation is more complicated. In this article, the input multimodal data is collected from Surrey Audio-Visual Expressed Emotion (SAVEE) and YouTube datasets, and then hybrid feature extraction is performed to extract feature vectors from the different modalities like textual, audio and visual. The Latent Dirichlet Allocation (LDA) and Latent Semantic Analysis (LSA) techniques are applied to extract features from textual modality. Further, AlexNet, spectral centroid features, spectral flux features, and short term energy are employed to extract features from the visual and audio modalities. The extracted active feature values are given as the input to the Modified Ant Lion Optimizer (MALO) based Long Short Term Memory (LSTM) network for sentiment classification. In the MALO algorithm, two new processes are performed to select optimal hyper-parameters of LSTM network that improves the training and testing mechanism and reduces the computational complexity. The results with the MALO-LSTM model obtained were 98.62% and 98.81% of accuracy on YouTube and SAVEE datasets, relatively higher than some of the existing methods, the proposed approach provided comparatively better performance.

show abstract

Multi-level context extraction and attention-based contextual inter-modal fusion for multimodal sentiment analysis and emotion classification

Cited by 29 publications

References 33 publications

Neuronal Apoptosis in Patients with Liver Cirrhosis and Neuronal Epileptiform Discharge Model Based upon Multi-Modal Fusion Deep Learning

Neuronal Apoptosis in Patients with Liver Cirrhosis and Neuronal Epileptiform Discharge Model Based upon Multi-Modal Fusion Deep Learning

[Retracted] Music Emotion Research Based on Reinforcement Learning and Multimodal Information

MALO-LSTM: Multimodal Sentiment Analysis Using Modified Ant Lion Optimization with Long Short Term Memory Network

Contact Info

Product

Resources

About