Multimodal Mild Depression Recognition Based on EEG-EM Synchronization Acquisition Network

Abstract

In this paper, we used electroencephalography (EEG)-eye movement (EM) synchronization acquisition network to simultaneously record both EEG and EM physiological signals of the mild depression and normal controls during free viewing. Then, we consider a multimodal feature fusion method that can best discriminate between mild depression and normal control subjects as a step toward achieving our longterm aim of developing an objective and effective multimodal system that assists doctors during diagnosis and monitoring of mild depression. Based on the multimodal denoising autoencoder, we use two feature fusion strategies (feature fusion and hidden layer fusion) for fusion of the EEG and EM signals to improve the recognition performance of classifiers for mild depression. Our experimental results indicate that the EEG-EM synchronization acquisition network ensures that the recorded EM and EEG data require that both the data streams are synchronized with millisecond precision, and both fusion methods can improve the mild depression recognition accuracy, thus demonstrating the complementary nature of the modalities. Compared with the unimodal classification approach that uses only EEG or EM, the feature fusion method slightly improved the recognition accuracy by 1.88%, while the hidden layer fusion method significantly improved the classification rate by up to 7.36%. In particular, the highest classification accuracy achieved in this paper was 83.42%. These results indicate that the multimodal deep learning approaches with input data using a combination of EEG and EM signals are promising in achieving real-time monitoring and identification of mild depression.

Exploration of EEG-Based Depression Biomarkers Identification Techniques and Their Applications: A Systematic Review

“…Frontal (FP1, FP2, FPz) [68], [82], [100], [70], [ Not Mentioned [47], [48], [43] Most of these studies used EEG data from the individual modality and used a large number of electrodes. Out of 52 articles, 19 research articles have been found where authors have used 19 electrodes EEG device and the electrode positions for the 19-channel EEG are Fp1, Fp2, F3, F4, F7, F8, C3, C4, T3, T4, T5, T6, P3, P4, O1, O2, Fz, Cz, Pz.…”

“…However, as we have mentioned earlier, they used a certain stimulus where the authors used facial expression to picture block along with the neutral block of that person for a certain time. Zhu et al used an eye tracker along with the EEG Recording [43].…”

“…Cutoff frequency/ Bandpass filters [61], [63], [42], [36], [117] , [48], [43], [87], [72], [88] Certain ranges of high Bandpass and low Bandpass filters are used to reduce undesired large signal distortions.…”

“…Pre-processed EEG Device data/ Builtin Plugins [60], [39], [22], [56], [84], [43], [74], [57], [65], [37], [49], [81] Some EEG (Muse) devices provide both raw and preprocessed EEG data; in that cases, the devices filter and allow signals with certain frequency ranges. Moreover, many software and toolboxes (such as-MATLAB EEGLAB epoch rejection function; Net Station Waveform; BESA; QExG; TrimOutlier) manage built-in plugins that are used for automatic artefacts removal.…”

“…ICA/ FastICA/ FASTER algorithm [24], [56], [62], [44], [36], [83], [32], [23], [37], [31], [58], [20], [44], [70], [86], [88] Used to reduce EOG or blinking and eye movement-related artefacts 50Hz Notch filter [55], [42], [43], [64], [61], [67], [47], [75], [20], [49], [50], [99], [88] Used to remove 50Hz power line noise Reduction of physical activities and Visual Expertise to avoid artifacts [48], [77], [53], [64], [78], [79] While recording the data, participants were instructed to close their eyes and reduce movement, swallowing saliva, and breathing normally to suppress the blinking artefacts as much as possible. [101], [111], [75] Kolmogorov entropy [68], [51], [113], [71] Shannon entropy [51], [53], [113], [71] Sample Entropy (SampEn) [99], <...>…”