EEG-Based Emotion Recognition Using Quadratic Time-Frequency Distribution

Abstract

Accurate recognition and understating of human emotions is an essential skill that can improve the collaboration between humans and machines. In this vein, electroencephalogram (EEG)-based emotion recognition is considered an active research field with challenging issues regarding the analyses of the nonstationary EEG signals and the extraction of salient features that can be used to achieve accurate emotion recognition. In this paper, an EEG-based emotion recognition approach with a novel time-frequency feature extraction technique is presented. In particular, a quadratic time-frequency distribution (QTFD) is employed to construct a high resolution time-frequency representation of the EEG signals and capture the spectral variations of the EEG signals over time. To reduce the dimensionality of the constructed QTFD-based representation, a set of 13 time- and frequency-domain features is extended to the joint time-frequency-domain and employed to quantify the QTFD-based time-frequency representation of the EEG signals. Moreover, to describe different emotion classes, we have utilized the 2D arousal-valence plane to develop four emotion labeling schemes of the EEG signals, such that each emotion labeling scheme defines a set of emotion classes. The extracted time-frequency features are used to construct a set of subject-specific support vector machine classifiers to classify the EEG signals of each subject into the different emotion classes that are defined using each of the four emotion labeling schemes. The performance of the proposed approach is evaluated using a publicly available EEG dataset, namely the DEAPdataset. Moreover, we design three performance evaluation analyses, namely the channel-based analysis, feature-based analysis and neutral class exclusion analysis, to quantify the effects of utilizing different groups of EEG channels that cover various regions in the brain, reducing the dimensionality of the extracted time-frequency features and excluding the EEG signals that correspond to the neutral class, on the capability of the proposed approach to discriminate between different emotion classes. The results reported in the current study demonstrate the efficacy of the proposed QTFD-based approach in recognizing different emotion classes. In particular, the average classification accuracies obtained in differentiating between the various emotion classes defined using each of the four emotion labeling schemes are within the range of 73.8%–86.2%. Moreover, the emotion classification accuracies achieved by our proposed approach are higher than the results reported in several existing state-of-the-art EEG-based emotion recognition studies.

Human Emotion Recognition: Review of Sensors and Methods

“…The EEG signals provided by the DEAP dataset [58] New deep learning framework based on a multiband feature matrix (MFM) and a capsule network (CapsNet) is proposed.…”

Investigating the Use of Pretrained Convolutional Neural Network on Cross-Subject and Cross-Dataset EEG Emotion Recognition

“…In [41], KNN method is employed on the DEAP dataset [42] for different numbers of channels which show accuracy between 82% and 88%. The study conducted in [43] quadratic time-frequency distribution (QTFD) is employed to handle a high-resolution time-frequency representation of the EEG and the spectral variations over time. It reports mean classification accuracies ranging between 73.8% and 86.2%.…”

“…In EEG data channels, typical frequency domain analysis is used. In the frequency domain, the most important frequency bands are delta (1-3 Hz), theta (4-7 Hz), alpha (8)(9)(10)(11)(12)(13), beta (14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29)(30) and gamma (31)(32)(33)(34)(35)(36)(37)(38)(39)(40)(41)(42)(43)(44)(45)(46)(47)(48)(49)(50) [26]. Fast Fourier Transform (FFT), Wavelet Transform (WT), eigenvector and autoregressive are the methods which transform EEG signal from time domain to frequency domain [27].…”

Two Stepped Majority Voting for Efficient EEG based Emotion Classification

“…When Table 12 was examined, it was seen that the proposed method outperformed other methods on HV vs LV discrimination. Alzarzi et al produced a second-best accuracy score where the accuracy was 85.8% (Alazrai et al, 2018). Abeer et al, Tripathi et al and Li et al reported 82.0%, 81.4% and 80.7% accuracy scores, respectively (Al-Nafjan et al, 2017;Tripathi et al, 2017;Li et al, 2017).…”

“…The DEAP dataset was used in experiments and promising results were obtained. Alazrai et al proposed a methodology for EEG based emotion recognition (Alazrai et al, 2018). The authors used a new time-frequency (TF) based features for efficient classification.…”