Discriminating Fake and Real Smiles Using Electroencephalogram Signals With Convolutional Neural Networks

Abstract

Genuineness of smiles is of particular interest in the field of human emotions and social interactions. In this work, we develop an experimental protocol to elicit genuine and fake smile expressions on 28 healthy subjects. Then, we assess the type of smile expressions using electroencephalogram (EEG) signals with convolutional neural networks (CNNs). Five different architectures (CNN1, CNN2, CNN3, CNN4, and CNN5) were examined to differentiate between fake and real smiles. We transform the temporal EEG signals into normalized gray-scale images and perform three-way classification to classify fake smiles, genuine smiles, and neutral expressions in the form of subject-dependent classification. We achieved the highest classification accuracy of 90.4% using CNN1 for the full EEG spectrum. Likewise, we achieved classification accuracies of 87.4%, 88.3%, 89.7%, and 90.0% using Beta, Alpha, Theta, and Delta EEG bands respectively. This paper suggests that CNNs models, widely used in image classification problems, can provide an alternative approach for smile detection from physiological signals such as the EEG.

Electroencephalogram Analysis Method to Detect Unspoken Answers to Questions Using Multistage Neural Networks

“…Yang et al [44] validated their CNN model by comparison with k-NN, SVM, LDA, and other CNN models. Moussa et al [51] demonstrated the effectiveness of CNN models through comparison with LSTM, ANN and SVM. Similarly, Farokhah et al [52] validated the efficiency of the simplified CNN models.…”

“…EEG signals have been analyzed to create BCI systems using analysis techniques such as EEG feature extraction from an event-related potential (ERP) response [11], the power spectrum and spectral centroid [13][17]- [19], Bayesian-based EEG feature extraction techniques [20], principal component analysis [21]- [24], independent component analysis (ICA) [15] [23][25]- [27], EEG pattern classifiers, artificial neural networks (ANN) [14][23] [28], k-nearest neighbor algorithm [29] [30], linear discriminant analysis (LDA) [15][31]- [33], SVM [12][13] [21][34]- [37], self-organizing maps [38], and fuzzy entropy [39]. Furthermore, over the past few years, DNN, its improved models [7][8][40]- [44], and convolutional neural networks (CNN) [9][10][45]- [51] have been employed for EEG feature extraction and pattern classification. In particular, EEGNET was proposed as the dedicated EEG analysis model based on artificial intelligence techniques [46][52]- [55].…”

“…Prior research on MI-EEG have employed various computational models for classifying motor imagery. For instance, Lu et al [7] and Sturm et al [8] employed deep neural network (DNN) models, while Yang et al [6], Amin et al [9], Tabar et al [10], Alazrai et al [50], and Moussa et al [51] favored convolutional neural network (CNN) models. Furthermore, EEGNET was employed by Lawhern et al [46], Zhu et al [55], and Strahnen et al [54].…”

“…3) Used multistage neural networks to separate EEG signals with obvious artifact noise related to blink from those without these noises and separately extract the features before classifying the responses. 4) Used CNNs to extract features and classify the question responses, with previous studies demonstrating the effectiveness of using CNN models for EEG feature extraction and noise removal [9], [10], [45]- [51], [56].…”