Driver Fatigue Detection Based on Facial Key Points and LSTM

Chen, Long; Xin, Guojiang; Liu, Yuling; Huang, Jie

doi:10.1155/2021/5383573

Cited by 44 publications

(16 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The convolutional neural network, named EM-CNN, was used to detect the states of the eyes and mouth from the ROI images. Similarly, Chen et al [4] used MTCNN to detect a human face: an open-source software library (DLIB) was used to locate facial landmarks to extract the fatigue feature vector of each frame, and the long short-term memory (LSTM) network was used to obtain a final fatigue feature value. So, it can be seen from the literatures mentioned above that in all the features, the characteristics of the eyes and the mouth are the most widely used, and the establishment of complex models and the large amount of data to be processed pose new challenges to the computing power of computers.…”

Section: Detection Based On Driver Informationmentioning

confidence: 99%

“…However, the initial signs of fatigue can be detected before a critical situation arises, and developing systems to automatically detect driver fatigue and advise drivers to take a break in time has received increased interest. There are many methods available to determine the drowsiness state of a driver [1][2][3][4], including detection based on vehicle information, the driver's information, and multi-information fusion.…”

Section: Introductionmentioning

confidence: 99%

“…The fusion of multiple features can greatly improve the accuracy and robustness of fatigue detection, and it is more suitable for changeable environments. However, it requires various information that is difficult to obtain, and a fusion model is difficult to establish [4].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Chen

Yan

Zhu

et al. 2022

Sensors

View full text Add to dashboard Cite

Fatigue driving has always received a lot of attention, but few studies have focused on the fact that human fatigue is a cumulative process over time, and there are no models available to reflect this phenomenon. Furthermore, the problem of incorrect detection due to facial expression is still not well addressed. In this article, a model based on BP neural network and time cumulative effect was proposed to solve these problems. Experimental data were used to carry out this work and validate the proposed method. Firstly, the Adaboost algorithm was applied to detect faces, and the Kalman filter algorithm was used to trace the face movement. Then, a cascade regression tree-based method was used to detect the 68 facial landmarks and an improved method combining key points and image processing was adopted to calculate the eye aspect ratio (EAR). After that, a BP neural network model was developed and trained by selecting three characteristics: the longest period of continuous eye closure, number of yawns, and percentage of eye closure time (PERCLOS), and then the detection results without and with facial expressions were discussed and analyzed. Finally, by introducing the Sigmoid function, a fatigue detection model considering the time accumulation effect was established, and the drivers’ fatigue state was identified segment by segment through the recorded video. Compared with the traditional BP neural network model, the detection accuracies of the proposed model without and with facial expressions increased by 3.3% and 8.4%, respectively. The number of incorrect detections in the awake state also decreased obviously. The experimental results show that the proposed model can effectively filter out incorrect detections caused by facial expressions and truly reflect that driver fatigue is a time accumulating process.

show abstract

Section: Detection Based On Driver Informationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Chen

Yan

Zhu

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Zhao et al [22] take the driver's head posture as a fatigue feature, count and analyse the head posture differences between different fatigue states, and then predict the driver's state. Chen et al [23] combine the facial features to predict the fatigue level.…”

Section: Related Workmentioning

confidence: 99%

Driver Fatigue Detection Method Based on Human Pose Information Entropy

Zhang

et al. 2022

Journal of Advanced Transportation

View full text Add to dashboard Cite

Driver fatigue detection (DFD) is an effective method to prevent traffic accidents. The existing research on DFD using facial features is an effective and noninvasive fatigue detection method. However, this approach is affected by facial occlusions (glasses, sunglasses, masks, etc.) and the large facial pose deformations in the extraction of effective fatigue features. In this paper, we introduce a novel DFD method using human pose information entropy. The method first estimates human pose from video sequences and then uses them as clues to extract multiple fatigue-related features which can reduce the influence of facial occlusion and head pose deformation. Information entropy and sliding window algorithm are applied to analyse and calculate sufficient consecutive video frames to obtain more robust and accurate fatigue-related values than by using a single frame. These information entropy values are combined resorting to the support vector machine (SVM) to recognize the driver fatigue state. Experimental results show that the method can achieve much higher accuracy and robustness, and the detection speed meets the requirements of real time.

show abstract

“…Linking both networks can help fully extract the spatiotemporal feature expression of facial actions and realize driver fatigue detection using long-term sequence actions in a video. Chen et al [25] proposed a method based on the key facial points and an LSTM, which is similar to the method proposed in this paper, but it is not suitable for multiangle face detection. The method proposed in this paper has made corresponding improvements in face detection and temporal feature learning networks and performs well for the purposes of this paper.…”

Section: Introductionmentioning

confidence: 99%

Truck Driver Fatigue Detection Based on Video Sequences in Open-Pit Mines

Wang

2021

Mathematics

View full text Add to dashboard Cite

Due to complex background interference and weak space–time connection, traditional driver fatigue detection methods perform poorly for open-pit truck drivers. For these issues, this paper presents a driver fatigue detection method based on Libfacedetection and an LRCN. The method consists of three stages: (1) using a face detection module with a tracking method to quickly extract the ROI of the face; (2) extracting and coding the features; (3) combining the coding model to build a spatiotemporal classification network. The innovation of the method is to utilize the spatiotemporal features of the image sequence to build a spatiotemporal classification model suitable for this task. Meanwhile, a tracking method is added to the face detection stage to reduce time expenditure. As a result, the average speed with the tracking method for face detection on video is increased by 74% in comparison with the one without the tracking method. Our best model adopts a DHLSTM and feature-level frame aggregation, which achieves high accuracy of 99.30% on the self-built dataset.

show abstract

Driver Fatigue Detection Based on Facial Key Points and LSTM

Cited by 44 publications

References 27 publications

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Driver Fatigue Detection Method Based on Human Pose Information Entropy

Truck Driver Fatigue Detection Based on Video Sequences in Open-Pit Mines

Contact Info

Product

Resources

About