Correlated Attention Networks for Multimodal Emotion Recognition

Jielin, Qiu,; Li, Xiaoyü; Hu, Kai

doi:10.1109/bibm.2018.8621129

Cited by 19 publications

(12 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That is because both MEI-PCANet and GoogLeNet model consider the tiny characteristics of the target, and the multilayer network is used to describe the target, which increases the detection accuracy. Comparing the original CAN model [54] , the proposed HVG-based sparse classification scheme has improved the recognition rate effectively, which indicates the efficiency of our proposed hierarchical visual cognition model. Original SRC model achieves the lowest recognition rate.…”

Section: Resultsmentioning

confidence: 75%

See 1 more Smart Citation

Emotion-Aware and Intelligent Internet of Medical Things Toward Emotion Recognition During COVID-19 Pandemic

Zhang

Liu

Tian

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

The Internet of Medical Things (IoMT) is a brand new technology of combining medical devices and other wireless devices to access to the healthcare management systems. This article has sought the possibilities of aiding the current Corona Virus Disease 2019 (COVID-19) pandemic by implementing machine learning algorithms while offering emotional treatment suggestion to the doctors and patients. The cognitive model with respect to IoMT is best suited to this pandemic as every person is to be connected and monitored through a cognitive network. However, this COVID-19 pandemic still remain some challenges about emotional solicitude for infants and young children, elderly, and mentally ill persons during pandemic. Confronting these challenges, this article proposes an emotion-aware and intelligent IoMT system, which contains information sharing, information supervision, patients tracking, data gathering and analysis, healthcare, etc. Intelligent IoMT devices are connected to collect multimodal data of patients in a surveillance environments. The latest data and inputs from official websites and reports are tested for further investigation and analysis of the emotion analysis. The proposed novel IoMT platform enables remote health monitoring and decision-making about the emotion, therefore greatly contribute convenient and continuous emotion-aware healthcare services during COVID-19 pandemic. Experimental results on some emotion data indicate that the proposed framework achieves significant advantage when compared with the some mainstream models. The proposed cognition-based dynamic technology is an effective solution way for accommodating a big number of devices and this COVID-19 pandemic application. The controversy and future development trend are also discussed.

show abstract

Section: Resultsmentioning

confidence: 75%

“…The proposed model is compared with many classical and recent algorithms: correlated attention network (CAN) [54] , motion energy image-based principal component analysis network (MEI-PCANet) [17] , appearance and motion deepnet (AMDN) model [55] , GoogLeNet in [56] and VGG16 in [57] as well as the model in [44] .…”

Section: Resultsmentioning

confidence: 99%

Emotion-Aware and Intelligent Internet of Medical Things Toward Emotion Recognition During COVID-19 Pandemic

Zhang

Liu

Tian

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

show abstract

“…This indicates that the three feature extraction techniques performed differently on the DEAP dataset by extracting different discriminatory features from it. The various best results obtained by each of the three modalities and features are state of the art and better than the results obtained in recent research studies [7,[33][34][35][36][37][38][39][40][77][78][79][80][81][82][83][84][85][86][87][88][89] that utilized the DEAP dataset. These results are also better than those reported by [34,[36][37][38] despite the trending deep learning approaches applied in those studies.…”

Section: Discussionmentioning

confidence: 78%

Recognition of Human Emotion using Radial Basis Function Neural Networks with Inverse Fisher Transformed Physiological Signals

Abayomi

Olugbara

Heukelman

2021

IJIE

View full text Add to dashboard Cite

Emotion is a complex state of human mind influenced by body physiological changes and interdependent external events thus making an automatic recognition of emotional state a challenging task. A number of recognition methods have been applied in recent years to recognize human emotion. The motivation for this study is therefore to discover a combination of emotion features and recognition method that will produce the best result in building an efficient emotion recognizer in an affective system. We introduced a shifted tanh normalization scheme to realize the inverse Fisher transformation applied to the DEAP physiological dataset and consequently performed series of experiments using the Radial Basis Function Artificial Neural Networks (RBFANN). In our experiments, we have compared the performances of digital image based feature extraction techniques such as the Histogram of Oriented Gradient (HOG), Local Binary Pattern (LBP) and the Histogram of Images (HIM). These feature extraction techniques were utilized to extract discriminatory features from the multimodal DEAP dataset of physiological signals. Experimental results obtained indicate that the best recognition accuracy was achieved with the EEG modality data using the HIM features extraction technique and classification done along the dominance emotion dimension. The result is very remarkable when compared with existing results in the literature including deep learning studies that have utilized the DEAP corpus and also applicable to diverse fields of engineering studies.

show abstract

“…Valence(%) Arousal(%) Tang-2017 [25] 83.82 ± 5.01 83.23 ± 2.61 Liu-2016 [26] 85.20 ± 4.47 80.50 ± 3.39 Liu-2019 [27] 85.62 ± 3.48 84.33 ± 2.25 Qiu-2018 [28] 86.45 ± / 84.79 ± / Yin-2021 [29] 90.45 ± 3.09 90.60 ± 2.62 Yang-2018 [30] 90.80 ± 3.08 91.03 ± 2.99 Liao-2020 [31] 91.95 ± / 93.06 ± / Ma-2019 [32] 92.30 ± 1.55 92.87 ± 2.11 Huang-2021 [15] 94.38 ± 2.61 94.72 ± 2.56 Cui-2020 [23] 96.65 ± 2.65 97.11 ± 2.01 MSBAM 98.89 ± 1.03 98.87 ± 0.92…”

Section: Methodsmentioning

confidence: 99%

Simultaneously exploring multi-scale and asymmetric EEG features for emotion recognition

Wu¹,

Xia²,

Zhang³

et al. 2021

Preprint

View full text Add to dashboard Cite

In recent years, emotion recognition based on electroencephalography (EEG) has received growing interest in the brain-computer interaction (BCI) field. The neuroscience researches indicate that the left and right brain hemispheres demonstrate differences under different emotional activities, which is an important principle for designing deep learning (DL) model for emotion recognition. Besides, the neural activities of emotions may occur at different time scales and duration, it is beneficial to use multiple kernels with different sizes to learn the temporal multi-scale features. Based on these two angles, we propose Multi-Scales Bi-hemispheric Asymmetric Model (MSBAM) based on a two-branch convolutional Neural Networks (CNN). We first use different kernel in two separate branch to extract distinct temporal representations, and then bi-hemispheric asymmetric feature extractor to learn asymmetric spatial features in each branch. Then, the features from the two branches are combined together for emotion state recognition. Evaluated on the public DEAP dataset, MSBAM yields accuracies over 98% on two-class classification for both valence and arousal states.

show abstract

Correlated Attention Networks for Multimodal Emotion Recognition

Cited by 19 publications

References 9 publications

Emotion-Aware and Intelligent Internet of Medical Things Toward Emotion Recognition During COVID-19 Pandemic

Emotion-Aware and Intelligent Internet of Medical Things Toward Emotion Recognition During COVID-19 Pandemic

Recognition of Human Emotion using Radial Basis Function Neural Networks with Inverse Fisher Transformed Physiological Signals

Simultaneously exploring multi-scale and asymmetric EEG features for emotion recognition

Contact Info

Product

Resources

About