Multi-Input CNN-LSTM deep learning model for fear level classification based on EEG and peripheral physiological signals

Abstract

Objective and accurate classification of fear levels is a socially important task that contributes to developing treatments for Anxiety Disorder, Obsessive–compulsive Disorder, Post-Traumatic Stress Disorder (PTSD), and Phobia. This study examines a deep learning model to automatically estimate human fear levels with high accuracy using multichannel EEG signals and multimodal peripheral physiological signals in the DEAP dataset. The Multi-Input CNN-LSTM classification model combining Convolutional Neural Network (CNN) and Long Sort-Term Memory (LSTM) estimated four fear levels with an accuracy of 98.79% and an F1 score of 99.01% in a 10-fold cross-validation. This study contributes to the following; (1) to present the possibility of recognizing fear emotion with high accuracy using a deep learning model from physiological signals without arbitrary feature extraction or feature selection, (2) to investigate effective deep learning model structures for high-accuracy fear recognition and to propose Multi-Input CNN-LSTM, and (3) to examine the model’s tolerance to individual differences in physiological signals and the possibility of improving accuracy through additional learning.

EEG‐based efficient demographic classification with selective data approach and scalable hybrid network

“…Simulation results: We evaluated our proposed method's effectiveness while classifying the multi-modal mobile brain-body imaging (MOBI) dataset, comprising EEG data from 88 participants recorded while playing Minecraft [20]. Our model was compared with other evaluated models such as Ravindran et al [20], fused CNN-LSTM [21], and multiinput CNN-LSTM [22]. All results were analysed using a tenfold crossvalidation (CV) method.…”

CNN-Based Multi-Factor Authentication System for Mobile Devices Using Faces and Passwords

“…The image itself was encrypted and transmitted to the server to increase the security level, and the data of 60 people were classified with 99.33% accuracy. Utilizing a CNN-LSTM (long short-term memory) structure for fear level classification, Masuda and Yairi [48] learned electroencephalogram (EEG) and peripheral physiological signal (PPS) data. Feature-level concatenation with a parallel structure was employed, and four fear levels were distinguished with an accuracy of 98.79% using CNN-LSTM with 11 layers.…”

“…The system of Rajasekar et al [41] recognizes only the image as a factor and, so, it cannot implement multi-modality. The system of Masuda and Yairi [48] has 11 layers, making it unsuitable for mobile devices, and does not consider data safety. Sasikala [49] proposed a relatively lightweight system with six layers, but it is a complex system with a convolutional layer, gated recurrent unit (GRU) layer, and attention layer; furthermore, it implements only image data, so it cannot implement multi-modality.…”

A comprehensive review for machine learning on neuroimaging in obsessive-compulsive disorder

“…In a study conducted in 2023, an EEG-based approach was employed, integrating Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks in a multi-input CNN-LSTM architecture. This deep learning model demonstrated high accuracy in identifying fear emotions from physiological signals(Masuda and Yairi, 2023). In another study utilizing resting-state functional MRI (rs-fMRI), data from 91 individuals diagnosed with post-traumatic stress disorder and 126 trauma-exposed individuals without PTSD were included.…”