A Regression Method With Subnetwork Neurons for Vigilance Estimation Using EOG and EEG

Abstract

In recent years, it has been observed that there is an increasing rate of road accidents due to the low vigilance of drivers. Thus, the estimation of drivers' vigilance state plays a significant role in Public Transportation Safety (PTS). We have adopted a feature fusion strategy that combines the electroencephalogram (EEG) signals collected from various sites of the human brain, including forehead, temporal, and posterior and forehead electrooculogram (forehead-EOG) signals, to address this factor. The level of vigilance is predicted through a new learning model known as double-layered neural network with subnetwork nodes (DNNSN), which comprises several subnetwork nodes, and each node in turn is composed of many hidden nodes that have various capabilities of feature selection (dimension reduced), feature learning, etc. The proposed single modality that uses only forehead-EOG signal exhibits a mean root-mean-square error (RMSE) of 0.12 and a mean Pearson product-moment correlation coefficient (COR) of 0.78. On one hand, an EEG signal achieved a mean RMSE of 0.13 and a mean COR of 0.72. Whereas, on the other, the proposed multimodality achieved values of 0.09 and 0.85 for the mean RMSE and the mean COR, respectively. Experimental results show that the proposed DNNSN with multimodality fusion outperforms the model with single modality for vigilance estimation due to the complementary information between forehead-EOG and EEG. After a favorable learning rate was applied to the input layer, the mean RMSE/COR improved to 0.11/0.79, 0.12/0.74, and 0.08/0.86, respectively. Hence, this quantitative analysis proves that the proposed method provides better feasibility and efficiency learning capability and surmounts other state-of-theart techniques.

Deep Coupling Recurrent Auto-Encoder with Multi-Modal EEG and EOG for Vigilance Estimation

“…In this study, eight feature level fusion methods were selected for comparison, including SVR, CCNF and CCRF [2], DAE [3], GELM [15], LSTM [16], DNNSN [17], and LSTM-CapsAtt [14]. The DNNSN proposes a double-layer neural network with subnetwork nodes.…”

“…This method explores time-dependent information and significantly improves the performance of vigilance estimation. The authors of [17] proposed a double-layered neural network with subnetwork nodes (DNNSN), which is composed of several subnet nodes, and each node is composed of many hidden nodes with various feature selection capabilities. Zhang [14] suggested that the capsule attention model and deep LSTM should be integrated with EEG and EEG.…”

Capsule Attention for Multimodal EEG-EOG Representation Learning With Application to Driver Vigilance Estimation

“…In [13], the use of an LSTM network resulted in a considerable improvement using feature fusion over single-mode EEG and EOG. In [12], a Doublelayered Neural Network with Subnetwork Nodes (DNNSN) was utilized along with multimodal feature selection using an autoencoder, obtaining impressive results. In [29], two domain adaption networks, notably Domain-Adversarial Neural Network (DANN) and Adversarial Discriminative Domain Adaptation (ADDA) were employed with feature fusion.…”

“…Moreover, EEG and EOG are susceptible to artifacts caused by motion and muscle activity such as jaw motion, frowning, and others, making their interpretation particularly challenging [14], [15]. Lastly, multimodal analysis of biological signals is generally difficult since identifying the complementary and contradicting information in the available signals is a challenging [12].…”

Distilling EEG Representations via Capsules for Affective Computing

“…A variety of deep learning techniques have been used to learn the most discriminative features extracted from EEG for affective computing. For example, In [7], the authors adopted Double-Layered Neural Network with Subnetwork Nodes (DNNSN) to estimate drivers' vigilance levels . In [8], a Graph regularized Extreme Learning Machine (GELM) was employed to predict fatigue .…”