Driving Stress Detection Using Multimodal Convolutional Neural Networks with Nonlinear Representation of Short-Term Physiological Signals

Lee, Jaewon; Lee, Hyeon-Jeong; Shin, Moo Whan

doi:10.3390/s21072381

Cited by 29 publications

(14 citation statements)

References 39 publications

(57 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The suggested technique beats state-of-the-art models on driver stress detection, with an average accuracy of 95.5%. Whereas in reference [47], the authors develop a multimodal CNN by using hand/foot galvanic skin response, and heart rate (HR) short-term input signals. The suggested technique achieved 92.33% of classification accuracy.…”

Section: Literature Reviewmentioning

confidence: 99%

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Riquelme¹,

Olivares²,

Mu駉z³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Traffic accidents are caused by driver fatigue or distraction in many cases. To prevent accidents, several low-cost hypovigilance (hypo-V) systems were developed in the past based on a multimodal-hybrid (physiological and behavioral) feature set. Similarly in this paper, real-time driver inattention and fatigue (Hypo-Driver) detection system is proposed through multi-view cameras and biosignal sensors to extract hybrid features. The considered features are derived from non-intrusive sensors that are related to the changes in driving behavior and visual facial expressions. To get enhanced visual facial features in uncontrolled environment, three cameras are deployed on multiview points (0 • , 45 • , and 90 • ) of the drivers. To develop a Hypo-Driver system, the physiological signals (electroencephalography (EEG), electrocardiography (ECG), electro-myography (sEMG), and electrooculography (EOG)) and behavioral information (PERCLOS70-80-90%, mouth aspect ratio (MAR), eye aspect ratio (EAR), blinking frequency (BF), head-titled ratio (HT-R)) are collected and pre-processed, then followed by feature selection and fusion techniques. The driver behaviors are classified into five stages such as normal, fatigue, visual inattention, cognitive inattention, and drowsy. This improved hypo-Driver system utilized trained behavioral features by a convolutional neural network (CNNs), recurrent neural network and long short-term memory (RNN-LSTM) model is used to extract physiological features. After fusion of these features, the Hypo-Driver system is classified hypo-V into five stages based on trained layers and dropout-layer in the deep-residual neural network (DRNN) model. To test the performance of a hypo-Driver system, data from 20 drivers are acquired. The results of Hypo-Driver compared to state-of-theart methods are presented. Compared to the state-of-the-art Hypo-V system, on average, the Hypo-Driver system achieved a detection accuracy (AC) of 96.5%. The obtained results indicate that the Hypo-Driver system based on multimodal and multiview features outperforms other state-of-the-art driver Hypo-V systems by handling many anomalies.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Riquelme¹,

Olivares²,

Mu駉z³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…These models also provide noise robustness and improved classification accuracy [29]. Various researcher including Lee et al [37], Hajinoroozi et al [38], Yan et al [39], and Rastgoo et al [15] have proposed Convolution Neural Network (CNN) based-based models for driver's state detection using different modalities. However, large deep learning models face gradient exploding or vanishing problems.…”

Section: Introductionmentioning

confidence: 99%

ECG-Based Driver’s Stress Detection Using Deep Transfer Learning and Fuzzy Logic Approaches

et al. 2022

View full text Add to dashboard Cite

Driver's stress detection is a critical research area to help reduce the likelihood of traffic accidents and driver's health complexities due to prolonged stress. Previous work in this area is heavily based on traditional machine learning models to classify the driver's stress levels using handcrafted features extraction techniques. Extracting the best features using these approaches is always a challenging task. Recently, deep learning techniques have emerged for constructing reliable features automatically and classifying the classes with high accuracy. However, large deep learning models face gradient exploding or vanishing problems. Moreover, acquiring a large dataset for training an entire network from scratch is also a challenging task. This paper is based on the deep transfer learning technique to avoid these problems and to reduce computational cost and time. Seven models are proposed for real-world driver's stress levels detection using Electrocardiogram (ECG) signals. Different Convolutional Neural Network (CNN)-based pre-trained networks are used to classify the driver's three stress levels. The time-frequency ECG components for the three stress levels are obtained as scalogram images using a normalized Continuous Wavelet Transform (CWT) filter bank and Morse wavelet. Results show that Model 5 based on Xception outperforms the GoogLeNet, InceptionResNetV2, and InceptionV3 based models by 11.32%, 11.32%, 9.45%, 7.54%, 5.66%, and 1.88% respectively and achieves 98.11% overall validation accuracy. Ranking estimation using fuzzy logic approach shows that Xception based Model 5 achieves the highest rank for driver's high and medium stress levels, while DenseNet-201 based Model 4 achieves the highest rank for low-stress level detection among the other models.

show abstract

“…The deployment of modern wireless communication systems, based on spectrally efficient modulation schemes such as the orthogonal frequency division multiplexing (OFDM), is perhaps the main agent that has pushed the dawn of new ultra-linear and highly efficient power amplifiers (PAs) [ 1 ]. Advanced techniques in the discrete-time domain have been employed to quantitatively mitigate the nonlinear impairments generated by the PA and the IQ modulator, mainly through digital predistorters (DPD) in transmitters linearization [ 2 , 3 , 4 , 5 ], and also by applying post-compensation in the communications receiver [ 6 , 7 ].…”

Section: Introductionmentioning

confidence: 99%

Upgrading Behavioral Models for the Design of Digital Predistorters

Crespo‐Cadenas

Madero‐Ayora

Becerra

2021

Sensors

View full text Add to dashboard Cite

This work presents a strategy to upgrade models for power amplifier (PA) behavioral modeling and digital predistortion (DPD). These incomplete structures are the consequence of nonlinear order and memory depth model truncation with the purpose of reducing the demand of the limited computational resources available in standard processors. On the other hand, the alternative use of model structures pruned a priori does not guarantee that every significant term is included. To improve the limited performance of an incomplete model, a general procedure to augment its structure by incorporating significant terms is demonstrated. The sparse nature of the problem allows a successive search incorporating additional terms with higher nonlinear order and memory depth. This approach is investigated in the modeling and linearization of a commercial class AB PA operating at a compression point of about 6 dB, and a class J PA operating near saturation. Results highlight the capabilities of this upgrading procedure in the improvement of linearization capabilities of DPDs.

show abstract

Driving Stress Detection Using Multimodal Convolutional Neural Networks with Nonlinear Representation of Short-Term Physiological Signals

Cited by 29 publications

References 39 publications

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

ECG-Based Driver’s Stress Detection Using Deep Transfer Learning and Fuzzy Logic Approaches

Upgrading Behavioral Models for the Design of Digital Predistorters

Contact Info

Product

Resources

About