EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks

Abstract

Driver fatigue detection is one of the essential tools to reduce accidents and improve traffic safety. Its main challenge lies in the problem of how to identify the driver's fatigue state accurately. Existing detection methods include yawning and blinking based on facial expressions and physiological signals. Still, lighting and the environment affect the detection results based on facial expressions. In contrast, the electroencephalographic (EEG) signal is a physiological signal that directly responds to the human mental state, thus reducing the impact on the detection results. This paper proposes a log-Mel spectrogram and Convolution Recurrent Neural Network (CRNN) model based on EEG to implement driver fatigue detection. This structure allows the advantages of the different networks to be exploited to overcome the disadvantages of using them individually. The process is as follows: first, the original EEG signal is subjected to a one-dimensional convolution method to achieve a Short Time Fourier Transform (STFT) and passed through a Mel filter bank to obtain a logarithmic Mel spectrogram, and then the resulting logarithmic Mel spectrogram is fed into a fatigue detection model to complete the fatigue detection task for the EEG signals. The fatigue detection model consists of a 6-layer convolutional neural network (CNN), bi-directional recurrent neural networks (Bi-RNNs), and a classifier. In the modeling phase, spectrogram features are transported to the 6-layer CNN to automatically learn high-level features, thereby extracting temporal features in the bi-directional RNN to obtain spectrogram-temporal information. Finally, the alert or fatigue state is obtained by a classifier consisting of a fully connected layer, a ReLU activation function, and a softmax function. Experiments were conducted on publicly available datasets in this study. The results show that the method can accurately distinguish between alert and fatigue states with high stability. In addition, the performance of four existing methods was compared with the results of the proposed method, all of which showed that the proposed method could achieve the best results so far.

A Novel Approach for Automatic Detection of Driver Fatigue Using EEG Signals Based on Graph Convolutional Networks

“…The accuracy of multi-level classification in this study is reported to be above 89%. Gao et al [ 21 ] presented a new model based on a logarithmic spectrogram and convolutional-RNN based on EEG signals. The proposed architecture of these researchers included six convolutional layers with a two-way RNN.…”

Scaling effectivity in manifold methodologies to detect driver’s fatigueness and drowsiness state

“…Further, it is seen that there are varied studies which considers hybrid techniques in order to increase the possibilities of accomplishing better accuracies. Hybrid techniques are noted to combine band power of EEG and PERCLOS [26]. Further, spectral power of EEG is combined with ECG [27] while work done in [28] integrates sample entropy, spectral component of EEG, and entropy of ECG.…”

Development of Speech Command Control Based TinyML System for Post-Stroke Dysarthria Therapy Device

“…4 is the downsampling with logarithmic scale of spectrogram (log-spectrogram) was employed, using average pooling and pixel reduction with a 1x6 kernel was intended to reduce the number of initial pixels of the spectrogram to 99x43 pixels as the primary input to the CNN model. This preprocessing method was conducted from previous research and has significant result to make spectrogram feature convertible for small form factor [79][80][81][82][83][84][85].…”