Driver Drowsiness Detection based on Multimodal using Fusion of Visual-feature and Bio-signal

Choi, Hyungtak; Back, Moon-Ki; Lee, Kyu-Chul

doi:10.1109/ictc.2018.8539478

Cited by 21 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the training process is performed by using Hypo-DB as mentioned in sub-section 3.1. To perform comparisons with the other state-of-the-art hypovigilance detection systems, four studies are selected such as Du-RNN [18], Li-CNN [20], Chen-SBL [29], and Choi-LSTM [34]. To evaluate these multimodal based systems using machine-learning or deep learning algorithms, we have used the same techniques as implemented in the corresponding research papers.…”

Section: Resultsmentioning

confidence: 99%

“…We use sparse Bayesian learning (SBL) to detect fatigue-level and the PCA technique is also used to get optimal features. The LSTM model is utilized in Choi-LSTM [34] to make comparisons with the proposed Hypo-Driver system using behavioral and physiological signals.…”

Section: Resultsmentioning

confidence: 99%

“…A deep LSTM model is utilized in reference [33] to classify driver fatigue after extraction features through EEG and EOG signals. An LSTM model is also utilized in [34] to recognize driver drowsiness using behavioral and physiological signals. Also, in reference [35][36][37][38][39], the authors use multimodal features using behavioral and physiological features to detect hypovigilance.…”

Section: Literature Reviewmentioning

confidence: 99%

See 2 more Smart Citations

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Riquelme¹,

Olivares²,

Mu駉z³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Traffic accidents are caused by driver fatigue or distraction in many cases. To prevent accidents, several low-cost hypovigilance (hypo-V) systems were developed in the past based on a multimodal-hybrid (physiological and behavioral) feature set. Similarly in this paper, real-time driver inattention and fatigue (Hypo-Driver) detection system is proposed through multi-view cameras and biosignal sensors to extract hybrid features. The considered features are derived from non-intrusive sensors that are related to the changes in driving behavior and visual facial expressions. To get enhanced visual facial features in uncontrolled environment, three cameras are deployed on multiview points (0 • , 45 • , and 90 • ) of the drivers. To develop a Hypo-Driver system, the physiological signals (electroencephalography (EEG), electrocardiography (ECG), electro-myography (sEMG), and electrooculography (EOG)) and behavioral information (PERCLOS70-80-90%, mouth aspect ratio (MAR), eye aspect ratio (EAR), blinking frequency (BF), head-titled ratio (HT-R)) are collected and pre-processed, then followed by feature selection and fusion techniques. The driver behaviors are classified into five stages such as normal, fatigue, visual inattention, cognitive inattention, and drowsy. This improved hypo-Driver system utilized trained behavioral features by a convolutional neural network (CNNs), recurrent neural network and long short-term memory (RNN-LSTM) model is used to extract physiological features. After fusion of these features, the Hypo-Driver system is classified hypo-V into five stages based on trained layers and dropout-layer in the deep-residual neural network (DRNN) model. To test the performance of a hypo-Driver system, data from 20 drivers are acquired. The results of Hypo-Driver compared to state-of-theart methods are presented. Compared to the state-of-the-art Hypo-V system, on average, the Hypo-Driver system achieved a detection accuracy (AC) of 96.5%. The obtained results indicate that the Hypo-Driver system based on multimodal and multiview features outperforms other state-of-the-art driver Hypo-V systems by handling many anomalies.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Riquelme¹,

Olivares²,

Mu駉z³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…However, recent advances in the field of deep learning have prompted some authors to study algorithms for recognizing drowsiness by means of "data fusion", specifically, by processing both visual and physiological data coming from the car-driver. In this sense, an interesting work has been published in the work of [14]. The authors in this work proposed a system based on multimodal deep learning that recognizes both visual and physiological changes in the state of attention of the driver.…”

Section: Introductionmentioning

confidence: 99%

Ad-Hoc Shallow Neural Network to Learn Hyper Filtered PhotoPlethysmoGraphic (PPG) Signal for Efficient Car-Driver Drowsiness Monitoring

2019

View full text Add to dashboard Cite

In next-generation cars, safety equipment related to assisted driving systems commonly known as ADAS (advanced driver-assistance systems) are of particular interest for the major car-makers. When we talk about the “ADAS system”, we mean the devices and sensors having the precise objective of improving and making car driving safer, and among which it is worth mentioning rain sensors, the twilight sensor, adaptive cruise control, automatic emergency braking, parking sensors, automatic signal recognition, and so on. All these devices and sensors are installed on the new homologated cars to minimize the risk of an accident and make life on board of the car easier. Some sensors evaluate the movement and the opening of the eyes, the position of the head and its angle, or some physiological signals of the driver obtainable from the palm of the hands placed in the steering. In the present contribution, the authors will present an innovative recognition and monitoring system of the driver’s attention level through the study of the photoplethysmographic (PPG) signal detectable from the palm of the driver’s hands through special devices housed in the steering of the car. Through a particular and innovative post-processing algorithm of the PPG signal through a hyper-filtering framework, then processed by a machine learning framework, the entire pipeline proposed will be able to recognize and monitor the attention level of the driver with high accuracy and acceptable timing.

show abstract

“…Among various sensors, cameras are widely used in drowsiness driving detection due to properties of low cost and non-invasive. Meanwhile, the recent development of machine learning algorithms such as deep convolutional neural networks [8], [9] and stacked autoencoder [10] advance the research of the drowsiness detection model. Although current vision-based methods show great performance [11] in drowsiness driving detection, most of them are significantly depend on illumination conditions, and cannot fundamentally solve this problem due to the physical limitation of sensors.…”

mentioning

confidence: 99%

EDDD: Event-Based Drowsiness Driving Detection Through Facial Motion Analysis With Neuromorphic Vision Sensor

et al. 2020

View full text Add to dashboard Cite

Drowsiness driving is a principal factor of many fatal traffic accidents. This paper presents the first eventbased drowsiness driving detection (EDDD) system by using the recently developed neuromorphic vision sensor. Compared with traditional frame-based cameras, neuromorphic vision sensors, such as Dynamic Vision Sensors (DVS), have a high dynamic range and do not acquire full images at a fixed frame rate but rather have independent pixels that output intensity changes (called events) asynchronously at the time they occur. Since events are generated by moving edges in the scene, DVS is considered as an efficient and effective detector for the drowsiness driving-related motions. Based on this unique output, this work first proposes a highly efficient method to recognize and localize the driver's eyes and mouth motions from event streams. We further design and extract event-based drowsiness-related features directly from the event streams caused by eyes and mouths motions, then the EDDD model is established based on these features. Additionally, we provide the EDDD dataset, the first public dataset dedicated to event-based drowsiness driving detection. The EDDD dataset has 260 recordings in daytime and evening with several challenging scenes such as subjects wearing glasses/sunglasses. Experiments are conducted based on this dataset and demonstrate the high efficiency and accuracy of our method under different illumination conditions. As the first investigation of the usage of DVS in drowsiness driving detection applications, we hope that this work will inspire more event-based drowsiness driving detection research.

show abstract

Driver Drowsiness Detection based on Multimodal using Fusion of Visual-feature and Bio-signal

Cited by 21 publications

References 7 publications

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Ad-Hoc Shallow Neural Network to Learn Hyper Filtered PhotoPlethysmoGraphic (PPG) Signal for Efficient Car-Driver Drowsiness Monitoring

EDDD: Event-Based Drowsiness Driving Detection Through Facial Motion Analysis With Neuromorphic Vision Sensor

Contact Info

Product

Resources

About