Emotion classification on eye-tracking and electroencephalograph fused signals employing deep gradient neural networks

Wu, Qun; Dey, Nilanjan; Shi, Fuqian; Crespo, Rubén González; Sherratt, R. Simon

doi:10.1016/j.asoc.2021.107752

Cited by 25 publications

(11 citation statements)

References 47 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Many previous studies have reported the differences between healthy and abnormal groups in frequency domain [46,47]. To explore the differences in network patterns be-tween DE and HC groups in frequency bands, the preprocessed EEG signals were firstly filtered into delta (1-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), beta (13-30 Hz), and gamma (30-50 Hz) bands using a Chebyshev filter [48][49][50]. The band-pass filtering process involves first passing through a low-pass filter and then through a high-pass filter.…”

Section: Eeg Network Analysismentioning

confidence: 99%

EEG Network Analysis of Depressive Emotion Interference Spatial Cognition Based on a Simulated Robotic Arm Docking Task

Yang,

Hu,

Zeng

et al. 2023

Brain Sciences

View full text Add to dashboard Cite

Depressive emotion (DE) refers to clinically relevant depressive symptoms without meeting the diagnostic criteria for depression. Studies have demonstrated that DE can cause spatial cognition impairment. However, the brain network mechanisms underlying DE interference spatial cognition remain unclear. This study aimed to reveal the differences in brain network connections between DE and healthy control (HC) groups during resting state and a spatial cognition task. The longer operation time of the DE group during spatial cognition task indicated DE interference spatial cognition. In the resting state stage, the DE group had weaker network connections in theta and alpha bands than the HC group had. Specifically, the electrodes in parietal regions were hubs of the differential networks, which are related to spatial attention. Moreover, in docking task stages, the left frontoparietal network connections in delta, beta, and gamma bands were stronger in the DE group than those of the HC group. The enhanced left frontoparietal connections in the DE group may be related to brain resource reorganization to compensate for spatial cognition decline and ensure the completion of spatial cognition tasks. Thus, these findings might provide new insights into the neural mechanisms of depressive emotion interference spatial cognition.

show abstract

Section: Eeg Network Analysismentioning

confidence: 99%

EEG Network Analysis of Depressive Emotion Interference Spatial Cognition Based on a Simulated Robotic Arm Docking Task

Yang,

Hu,

Zeng

et al. 2023

Brain Sciences

View full text Add to dashboard Cite

show abstract

“…Another study on multimodal emotion recognition was conduced by López-Gil et al (2016) who found that combining different signal sources on the feature level enables the detection of self-regulatory behavior more effectively than only using EEG data. Most recently, Wu et al (2021) fused EEG and eye tracking data for emotion classification using effective deep learning for a gradient neural network. They report an 88% accuracy for the recognition of eight emotions.…”

Section: Eeg and Eye Tracking Based Mental State Detectionmentioning

confidence: 99%

Multimodal EEG and Eye Tracking Feature Fusion Approaches for Attention Classification in Hybrid BCIs

Vortmann

Majed

Putze

2022

Front. Comput. Sci.

View full text Add to dashboard Cite

Often, various modalities capture distinct aspects of particular mental states or activities. While machine learning algorithms can reliably predict numerous aspects of human cognition and behavior using a single modality, they can benefit from the combination of multiple modalities. This is why hybrid BCIs are gaining popularity. However, it is not always straightforward to combine features from a multimodal dataset. Along with the method for generating the features, one must decide when the modalities should be combined during the classification process. We compare unimodal EEG and eye tracking classification of internally and externally directed attention to multimodal approaches for early, middle, and late fusion in this study. On a binary dataset with a chance level of 0.5, late fusion of the data achieves the highest classification accuracy of 0.609–0.675 (95%-confidence interval). In general, the results indicate that for these modalities, middle or late fusion approaches are better suited than early fusion approaches. Additional validation of the observed trend will require the use of additional datasets, alternative feature generation mechanisms, decision rules, and neural network designs. We conclude with a set of premises that need to be considered when deciding on a multimodal attentional state classification approach.

show abstract

“…Dessa forma, estudos que envolvem a curva ROC, podem avaliar a capacidade de classificação em diferir duas classes de estímulos associativos, podendo neste contexto indicar a presença ou ausência de um estímulo. Revelando a taxa de verdadeiros positivos (sensibilidade ou Recall) em relação à taxa de falsos positivos (1 -especificidade) para diferentes limiares de classificação 7 . Sendo possível ainda, a utilização de uma interface Trigger-in para garantir a precisão temporal dos dados e a sincronização adequada entre os diferentes dispositivos na identificação de estímulos visuais, com a combinação do uso de eye-tracking e do EEG, fazendo com que o sinal de movimento dos olhos acione a marca no eletroencefalograma 7 .…”

Section: Introductionunclassified

“…A partir disso, podemos analisar o Recall e a especificidade dos sinais gerados 7 Os elementos do sistema de redes (Networks) decorrem da interação de seus componentes 9 . Os diferentes tipos de interação permitem a coexistência da atuação de diversas redes em paralelo ao "sistema principal".…”

Section: Introductionunclassified

Análise integrada de dados para compreensão dos processos visuais no córtex

Lourenço Nascimento

2023

Rev Neurocienc

View full text Add to dashboard Cite

A compreensão dos mecanismos de processamento visual é essencial para o estudo da percepção visual. Neste artigo, foi discutido as áreas corticais envolvidas no processamento visual, como o córtex visual primário (área V1) e as áreas visual secundária e associativa. Essas áreas desempenham papéis distintos na análise de características visuais, como forma, cor e movimento. Além disso, foi destacado a importância de diferentes abordagens para analisar dados de atividade neural e resposta a estímulos visuais. Conclui-se que a integração de diferentes técnicas de análise, como séries temporais, curva ROC e modelos de rede, pode ampliar o escopo de investigação dos processos de percepção visual. Essas abordagens fornecem insights valiosos sobre a organização e o funcionamento do córtex visual em diferentes níveis de processamento. A aplicação dessas técnicas em estudos de percepção visual pode contribuir para uma compreensão mais profunda dos mecanismos envolvidos na formação da experiência visual.

show abstract

Emotion classification on eye-tracking and electroencephalograph fused signals employing deep gradient neural networks

Cited by 25 publications

References 47 publications

EEG Network Analysis of Depressive Emotion Interference Spatial Cognition Based on a Simulated Robotic Arm Docking Task

EEG Network Analysis of Depressive Emotion Interference Spatial Cognition Based on a Simulated Robotic Arm Docking Task

Multimodal EEG and Eye Tracking Feature Fusion Approaches for Attention Classification in Hybrid BCIs

Análise integrada de dados para compreensão dos processos visuais no córtex

Contact Info

Product

Resources

About