Valence-Arousal Model based Emotion Recognition using EEG, peripheral physiological signals and Facial Expression

Zhu, Qionghua; Lu, Guanming; Yan, Jun

doi:10.1145/3380688.3380694

Cited by 14 publications

(6 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lew et al [14] proposed a regionally operated domain adversarial network (RODAN) to learn spatial-temporal relations, which reported 62.93%/ and 63.97% in two dimensions, respectively. Zhu et al [15] obtained 78.47% and 72.2% accuracy scores by fusing multimodal decision-level features through a 3D convolutional network. Alazrai et al [16] achieved the best result of 86.6% and 85.8% in valence and arousal dimension and 72.5% in four classes by constructing high-resolution representation in time-frequency domain with a quadratic time-frequency distribution (QTFD).…”

Section: Results On Deap Datasetmentioning

confidence: 99%

See 1 more Smart Citation

A feature‐based on potential and differential entropy information for electroencephalogram emotion recognition

Xie

Chai

et al. 2021

Electronics Letters

View full text Add to dashboard Cite

Electroencephalogram (EEG) signals by virtue of its good time resolution can provide a more comprehensive way for emotion recognition. To further mine the emotion-relevant information of EEG signals, a kind of innovative and effective feature named mean differential entropy (MDE) is proposed for EEG emotion recognition (EER). The MDE feature extracted from EEG signals supplements the additional potential information of each EEG frame on the basis of time-frequency information. To verify its effectiveness, extensive experiments are conducted on DEAP (database for emotion analysis using physiological signals) dataset. The experiment results of the proposed methods on public EEG emotion dataset DEAP reached accuracies of 94.56% and 91.84% in valence dimension and four emotion labels with gradient boosting decision tree (GBDT), and 95.15% in arousal dimension with Random Forest (RF), which proved that the MDE feature is helpful to improve the performance of EER. 62076094, Shanghai Science and Technology Program "Distributed and generative few-shot algorithm and theory research" under Grant No. 20511100600.

show abstract

Section: Results On Deap Datasetmentioning

confidence: 99%

“…Zhu et al. [15] obtained 78.47% and 72.2% accuracy scores by fusing multimodal decision‐level features through a 3D convolutional network. Alazrai et al.…”

Section: Results On Deap Datasetmentioning

confidence: 99%

A feature‐based on potential and differential entropy information for electroencephalogram emotion recognition

Xie

Chai

et al. 2021

Electronics Letters

View full text Add to dashboard Cite

show abstract

“…They achieved 80% accuracy for valence and 74% for arousal on the DEAP dataset using a subject-dependent strategy in a multimodal approach. In a similar study, Zhu et al ( 2020 ) used a weighted decision level fusion strategy for combining EEG, peripheral physiological signals, and facial expressions to recognize the arousal-valence state. They used a 3D convolutional neural network (CNN) to extract facial features and classify them, and they also used a 1D CNN to extract EEG features and classify them.…”

Section: Related Workmentioning

confidence: 99%

“…Combining different physiological signals for emotion recognition (Yazdani et al, 2012 ; Shu et al, 2018 ) or fusing only behavioral modalities have been widely explored (Busso et al, 2008 ; McKeown et al, 2011 ). Recently some studies tried to improve emotion recognition methods by exploiting both physiological and behavioral techniques (Zheng et al, 2018 ; Huang et al, 2019 ; Zhu et al, 2020 ). Many studies used a combination of facial expressions and EEG signals to achieve this improvement (Koelstra and Patras, 2013 ; Huang et al, 2017 ; Zhu et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

“…Recently some studies tried to improve emotion recognition methods by exploiting both physiological and behavioral techniques (Zheng et al, 2018 ; Huang et al, 2019 ; Zhu et al, 2020 ). Many studies used a combination of facial expressions and EEG signals to achieve this improvement (Koelstra and Patras, 2013 ; Huang et al, 2017 ; Zhu et al, 2020 ). Usually, these researchers work on data that has been collected from subjects while they are watching videos or looking at still images (Koelstra et al, 2011 ; Soleymani et al, 2011 ).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using Facial Micro-Expressions in Combination With EEG and Physiological Signals for Emotion Recognition

et al. 2022

View full text Add to dashboard Cite

Emotions are multimodal processes that play a crucial role in our everyday lives. Recognizing emotions is becoming more critical in a wide range of application domains such as healthcare, education, human-computer interaction, Virtual Reality, intelligent agents, entertainment, and more. Facial macro-expressions or intense facial expressions are the most common modalities in recognizing emotional states. However, since facial expressions can be voluntarily controlled, they may not accurately represent emotional states. Earlier studies have shown that facial micro-expressions are more reliable than facial macro-expressions for revealing emotions. They are subtle, involuntary movements responding to external stimuli that cannot be controlled. This paper proposes using facial micro-expressions combined with brain and physiological signals to more reliably detect underlying emotions. We describe our models for measuring arousal and valence levels from a combination of facial micro-expressions, Electroencephalography (EEG) signals, galvanic skin responses (GSR), and Photoplethysmography (PPG) signals. We then evaluate our model using the DEAP dataset and our own dataset based on a subject-independent approach. Lastly, we discuss our results, the limitations of our work, and how these limitations could be overcome. We also discuss future directions for using facial micro-expressions and physiological signals in emotion recognition.

show abstract

Delineating emotional differences between depressed and non-depressed individuals using a novel multimodal framework

Gill,

Singh,

Hooda

et al. 2024

Multimed Tools Appl

View full text Add to dashboard Cite

Valence-Arousal Model based Emotion Recognition using EEG, peripheral physiological signals and Facial Expression

Cited by 14 publications

References 19 publications

A feature‐based on potential and differential entropy information for electroencephalogram emotion recognition

A feature‐based on potential and differential entropy information for electroencephalogram emotion recognition

Using Facial Micro-Expressions in Combination With EEG and Physiological Signals for Emotion Recognition

Delineating emotional differences between depressed and non-depressed individuals using a novel multimodal framework

Contact Info

Product

Resources

About