A Novel Domain Adversarial Networks Based on 3D-LSTM and Local Domain Discriminator for Hearing-Impaired Emotion Recognition

Tian, Zekun; Li, Dahua; Yang, Yi; Hou, Fazheng; Yang, Zhiyi; Song, Yu; Gao, Qiang

doi:10.1109/jbhi.2022.3212475

Cited by 10 publications

(1 citation statement)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with hearing-impaired subjects, the emotion classification effect of non-hearing-impaired subjects is better. It can be speculated that this result may be related to the difficulty of facial emotion recognition of hearing-impaired subjects under the influence of physiological factors and social environment factors, which is the same as the conclusion of previous researchers [ 30 , 31 ].…”

Section: Resultssupporting

confidence: 83%

Image-Evoked Emotion Recognition for Hearing-Impaired Subjects with EEG Signals

Zhu

Jin

Bai

et al. 2023

Sensors

Self Cite

View full text Add to dashboard Cite

In recent years, there has been a growing interest in the study of emotion recognition through electroencephalogram (EEG) signals. One particular group of interest are individuals with hearing impairments, who may have a bias towards certain types of information when communicating with those in their environment. To address this, our study collected EEG signals from both hearing-impaired and non-hearing-impaired subjects while they viewed pictures of emotional faces for emotion recognition. Four kinds of feature matrices, symmetry difference, and symmetry quotient based on original signal and differential entropy (DE) were constructed, respectively, to extract the spatial domain information. The multi-axis self-attention classification model was proposed, which consists of local attention and global attention, combining the attention model with convolution through a novel architectural element for feature classification. Three-classification (positive, neutral, negative) and five-classification (happy, neutral, sad, angry, fearful) tasks of emotion recognition were carried out. The experimental results show that the proposed method is superior to the original feature method, and the multi-feature fusion achieved a good effect in both hearing-impaired and non-hearing-impaired subjects. The average classification accuracy for hearing-impaired subjects and non-hearing-impaired subjects was 70.2% (three-classification) and 50.15% (five-classification), and 72.05% (three-classification) and 51.53% (five-classification), respectively. In addition, by exploring the brain topography of different emotions, we found that the discriminative brain regions of the hearing-impaired subjects were also distributed in the parietal lobe, unlike those of the non-hearing-impaired subjects.

show abstract

Section: Resultssupporting

confidence: 83%