Deep Learning Network and Renyi-entropy Based Fusion Model for Emotion Recognition Using Multimodal Signals

Abstract

Emotion recognition is a significant research topic for interactive intelligence system with the wide range of applications in different tasks, like education, social media analysis, and customer service. It is the process of perceiving user's emotional response automatically to the multimedia information by means of implicit explanation. With initiation of speech recognition and the computer vision, research on emotion recognition with speech and facial expression modality has gained more popularity in recent decades. Due to non-linear polarity of signals, emotion recognition results a challenging task. To achieve facial emotion recognition using multimodal signals, an effective Bat Rider Optimization Algorithm (BROA)-based deep learning method is proposed in this research. However, the proposed optimization algorithm named BROA is derived by integrating Bat Algorithm (BA) with Rider Optimization Algorithm (ROA), respectively. Here, the multimodal signals include face image, EEG signals, and physiological signals such that the features extracted from these modalities are employed for the process of emotion recognition. The proposed method achieves better performance against exiting methods by acquiring maximum accuracy of 0.8794, and minimum FAR and minimum FRR of 0.1757 and 0.1806.

Designing an Improved VMD-CNNs-DS Algorithm for the Port Crane Bearing Fault Diagnosis

“…Therefore, the incorporation of an ensemble of classifiers can significantly enhance the accuracy and robustness of the system [11]. Due to their strong information fusion capability, CNN also exhibit great potential for "multi-information fusion" in recognition tasks [12,13]. In the field of rotating machinery fault signal recognition, Li et al proposed IDSCNN, a novel bearing fault diagnosis algorithm based on ensemble deep convolutional neural networks and an improved Dempster-Shafer based evidence fusion approach [14].…”

Aetiology on Emotion Recognition Strategies Using Video, Electroencephalogram and Physiological Signals

“…However, some emotions overlap, and common personal states are not well differentiated based on classification. In recent years, Deep Neural Networks (DNNs) have been introduced in emotion recognition, and their results show better performance compared to shallow techniques [5,7,13,51]. Moreover, several multimodal architectures have been designed to take advantage of the benefits of both approaches, which can be categorized into two classes: supervised and joint.…”