An Investigation of Deep Learning Models for EEG-Based Emotion Recognition

Abstract

Emotion is the human brain reacting to objective things. In real life, human emotions are complex and changeable, so research into emotion recognition is of great significance in real life applications. Recently, many deep learning and machine learning methods have been widely applied in emotion recognition based on EEG signals. However, the traditional machine learning method has a major disadvantage in that the feature extraction process is usually cumbersome, which relies heavily on human experts. Then, end-to-end deep learning methods emerged as an effective method to address this disadvantage with the help of raw signal features and time-frequency spectrums. Here, we investigated the application of several deep learning models to the research field of EEG-based emotion recognition, including deep neural networks (DNN), convolutional neural networks (CNN), long short-term memory (LSTM), and a hybrid model of CNN and LSTM (CNN-LSTM). The experiments were carried on the well-known DEAP dataset. Experimental results show that the CNN and CNN-LSTM models had high classification performance in EEG-based emotion recognition, and their accurate extraction rate of RAW data reached 90.12 and 94.17%, respectively. The performance of the DNN model was not as accurate as other models, but the training speed was fast. The LSTM model was not as stable as the CNN and CNN-LSTM models. Moreover, with the same number of parameters, the training speed of the LSTM was much slower and it was difficult to achieve convergence. Additional parameter comparison experiments with other models, including epoch, learning rate, and dropout probability, were also conducted in the paper. Comparison results prove that the DNN model converged to optimal with fewer epochs and a higher learning rate. In contrast, the CNN model needed more epochs to learn. As for dropout probability, reducing the parameters by ~50% each time was appropriate.

The NMT Scalp EEG Dataset: An Open-Source Annotated Dataset of Healthy and Pathological EEG Recordings for Predictive Modeling

“…Deep neural networks have received a lot of attention over the last decade and have been the primary tool of choice for automation in several application areas, including biomedical engineering. In EEG applications deep neural networks have been employed for emotion recognition (Zhang et al, 2020 ) and motor imagery classification (Wu et al, 2019 ). However, deep neural networks are known to be data hungry and require a significant amount of labeled data for training.…”

A Review on Signal Processing Approaches to Reduce Calibration Time in EEG-Based Brain–Computer Interface

“…Deep learning is a promising subfield of machine learning, which has successful applications in many fields, such as natural language processing and computer vision, etc ( Huang et al, 2021 ). Recently, it has attracted more attention in BCIs ( Dose et al, 2018 ; Ming et al, 2018 ; Liu Y. et al, 2020 ; Zhang Y. et al, 2020 ). It can build a unified end-to-end model directly applied to raw EEG signals.…”

Classification of Complex Emotions Using EEG and Virtual Environment: Proof of Concept and Therapeutic Implication

“…Recently, a growing body of research investigated emotionrecognition with deep learning algorithms. Particularly convolutional neural networks showed promising results, outperforming traditional ML models in distinguishing emotional states (Aloysius and Geetha, 2017;Li et al, 2018;Moon et al, 2018;Yang et al, 2019;Zhang et al, 2020). However, these models are still in their infancy and have various limitations for EEG-based BCI classification due to the limited training data available (Lotte et al, 2018).…”