Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences

Zhu, Tianjun; Zhang, Chuang; Wu, Tung-Lung; Ouyang, Zhuang; Li, Houzhi; Na, Xiaoxiang; Ji, Liang; Li, Weihao

doi:10.3390/app12042224

Cited by 41 publications

(12 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on the threshold value, we declare whether either eye is open or closed as in ''(2)''. As used in [45], if EAR > 30 is considered as open eyes, whereas if EAR < 0.20 for more than 2 seconds, it is considered as closed eye, thus triggering an alert as well as sending a notification to the admin through the cloud database along with a picture, time stamp, and real-time location from driver's mobile. Moreover, the admin can visualize driver data and trace the real-time location if needed.…”

Section: ) Eye Aspect Ratio (Ear)mentioning

confidence: 99%

IoT-Based Non-Intrusive Automated Driver Drowsiness Monitoring Framework for Logistics and Public Transport Applications to Enhance Road Safety

et al. 2023

View full text Add to dashboard Cite

The exponential growth in road accidents has led to a need for continuous driver monitoring to enhance road safety. Existing techniques rely on vehicle sensor-based and behavior analysis-based approaches, where the behavior analysis-based approaches are generally considered more desirable as they enable reliable detection of a more elaborate set of driver behaviors. They are categorized as intrusive and non-intrusive approaches. Unlike intrusive approaches that generally rely on constant direct human contact with sensors (physiological signals) and are sensitive to artifacts, non-intrusive approaches offer a more effective behavior monitoring using computer vision-based techniques. This paper proposes an end-to-end non-intrusive IoT-based automated framework to monitor driver behaviors, designed specifically for logistic and public transport applications. It consists of an embedded system, edge computing and cloud computing modules, and a mobile phone application, in an attempt to provide a holistic unified solution for drowsiness detection, monitoring, as well as evaluation of drivers. Drowsiness detection is based on detecting sleeping, yawning, and distraction behaviors using an image processing-based technique. To minimize the effects of latency, throughput, and packet losses, edge computing is performed using commercial off-the-shelf embedded boards. Moreover, a cloud-hosted real-time database for remote monitoring on interactive Android mobile application has been set up, where admin can add multiple drivers to get drowsiness notifications along with other useful related information for driver evaluation. An extensive experimental testing has been performed, obtaining encouraging results. An overall accuracy of 96% is achieved along with an enhanced robustness, portability, and usability of the proposed framework.INDEX TERMS ADAS, driver drowsiness, road safety, driver monitoring, IoT, edge computing, mobile application, road accidents, cloud computing.

show abstract

Section: ) Eye Aspect Ratio (Ear)mentioning

confidence: 99%

IoT-Based Non-Intrusive Automated Driver Drowsiness Monitoring Framework for Logistics and Public Transport Applications to Enhance Road Safety

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Similar to eye aspect ratio, mouth aspect ratio (MAR) [26] can be used to measure the opening and closing of the mouth. When the mouth is closed, MAR value tends toward 0, and as the mouth opens, MAR value gradually increases.…”

Section: Fatigue Detectionmentioning

confidence: 99%

Driver Attention Detection Based on Improved YOLOv5

2023

View full text Add to dashboard Cite

In response to negative impacts such as personal and property safety hazards caused by drivers being distracted while driving on the road, this article proposes a driver’s attention state-detection method based on the improved You Only Look Once version five (YOLOv5). Both fatigue and distracted behavior can cause a driver’s attention to be diverted during the driving process. Firstly, key facial points of the driver are located, and the aspect ratio of the eyes and mouth is calculated. Through the examination of relevant information and repeated experimental verification, threshold values for the aspect ratio of the eyes and mouth under fatigue conditions, corresponding to closed eyes and yawning, are established. By calculating the aspect ratio of the driver’s eyes and mouth, it is possible to accurately detect whether the driver is in a state of fatigue. Secondly, distracted abnormal behavior is detected using an improved YOLOv5 model. The backbone network feature extraction element is modified by adding specific modules to obtain different receptive fields through multiple convolution operations on the input feature map, thereby enhancing the feature extraction ability of the network. The introduction of Swin Transformer modules in the feature fusion network replaces the Bottleneck modules in the C3 module, reducing the computational complexity of the model while increasing its receptive field. Additionally, the network connection in the feature fusion element has been modified to enhance its ability to fuse information from feature maps of different sizes. Three datasets were created of distracting behaviors commonly observed during driving: smoking, drinking water, and using a mobile phone. These datasets were used to train and test the model. After testing, the mAP (mean average precision) has improved by 2.4% compared to the model before improvement. Finally, through comparison and ablation experiments, the feasibility of this method has been verified, which can effectively detect fatigue and distracted abnormal behavior.

show abstract

“…Facial fatigue symptoms include head nodding (head anomaly), yawning (mouth anomaly), and blink frequency (eye anomaly) [ 36 ]. However, due to the diversity of equipment in the online detection scene, factors such as viewing angle and the line of sight are likely to interfere with the detection of head movements.…”

Section: Our Proposed Approachmentioning

confidence: 99%

An Explainable Student Fatigue Monitoring Module with Joint Facial Representation

Lin

Tian

et al. 2023

Sensors

View full text Add to dashboard Cite

Online fatigue estimation is, inevitably, in demand as fatigue can impair the health of college students and lower the quality of higher education. Therefore, it is essential to monitor college students’ fatigue to diminish its adverse effects on the health and academic performance of college students. However, former studies on student fatigue monitoring are mainly survey-based with offline analysis, instead of using constant fatigue monitoring. Hence, we proposed an explainable student fatigue estimation model based on joint facial representation. This model includes two modules: a spacial–temporal symptom classification module and a data-experience joint status inferring module. The first module tracks a student’s face and generates spatial–temporal features using a deep convolutional neural network (CNN) for the relevant drivers of abnormal symptom classification; the second module infers a student’s status with symptom classification results with maximum a posteriori (MAP) under the data-experience joint constraints. The model was trained on the benchmark NTHU Driver Drowsiness Detection (NTHU-DDD) dataset and tested on an Online Student Fatigue Monitoring (OSFM) dataset. Our method outperformed the other methods with an accuracy rate of 94.47% under the same training–testing setting. The results were significant for real-time monitoring of students’ fatigue states during online classes and could also provide practical strategies for in-person education.

show abstract

Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences

Cited by 41 publications

References 32 publications

IoT-Based Non-Intrusive Automated Driver Drowsiness Monitoring Framework for Logistics and Public Transport Applications to Enhance Road Safety

IoT-Based Non-Intrusive Automated Driver Drowsiness Monitoring Framework for Logistics and Public Transport Applications to Enhance Road Safety

Driver Attention Detection Based on Improved YOLOv5

An Explainable Student Fatigue Monitoring Module with Joint Facial Representation

Contact Info

Product

Resources

About