The challenges of emotion recognition methods based on electroencephalogram signals: a literature review

Abstract

Electroencephalogram (EEG) signals in recognizing emotions have several advantages. Still, the success of this study, however, is strongly influenced by: i) the distribution of the data used, ii) consider of differences in participant characteristics, and iii) consider the characteristics of the EEG signals. In response to these issues, this study will examine three important points that affect the success of emotion recognition packaged in several research questions: i) What factors need to be considered to generate and distribute EEG data?, ii) How can EEG signals be generated with consideration of differences in participant characteristics?, and iii) How do EEG signals with characteristics exist among its features for emotion recognition? The results, therefore, indicate some important challenges to be studied further in EEG signals-based emotion recognition research. These include i) determine robust methods for imbalanced EEG signals data, ii) determine the appropriate smoothing method to eliminate disturbances on the baseline signals, iii) determine the best baseline reduction methods to reduce the differences in the characteristics of the participants on the EEG signals, iv) determine the robust architecture of the capsule network method to overcome the loss of knowledge information and apply it in more diverse data set.

Oversampling Approach Using Radius-SMOTE for Imbalance Electroencephalography Datasets

“…In addition to the EEG data oversampling process, there are several vital processes in emotion recognition, such as the feature extraction and the emotion classification processes using the Differential Entropy (DE) method. According to [42], the method is capable of characterizing spatial data from EEG signals with the highlights feature comprising foremost exact and steady features [28,[43][44][45][46]. The classification process in this study compares two methods, namely the Decision Tree and the Convolutional Neural Network.…”

“…The decomposition process is carried out by determining each frequency band's Low and High Pass values. Table 1 shows the respective Low Pass and Band Pass values for each frequency band [28,46,47]. In general, the EEG signal has five frequency bands, out of which four, namely Theta, Alpha, Beta, and Gamma, are correlated with emotional reactions [46,48].…”

“…Therefore, for the overall experiment, a total of 307200 DE feature data (7680 feature x 40 experiments) is obtained. The following is the formula for the Differential Entropy (DE) method [28,46]:…”

“…On the other hand, though emotion recognition accuracy has increased, the average accuracy is still below 85% for emotional arousal and valence. This problem becomes a challenge in future studies in its recognition based on EEG signals, which is strongly influenced by participant characteristics, such as personality traits, intellectual abilities, and gender [28,53]. Therefore, further study needs to be conducted to examine the oversampling approach and allow the combination with the baseline reduction approach to characterize participant characteristics.…”

“…Several studies have analyzed human emotion recognition, based on EEG signals, in data acquisition, feature extraction, and emotion classification [14,[23][24][25][26][27]. However, emotion recognition is strongly influenced by the distribution or balance of Electroencephalogram data [28]. Conversely, the limited data obtained significantly affects the imbalance condition of the resulting Electroencephalogram signal data.…”