Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Abstract

Fatigue driving has always received a lot of attention, but few studies have focused on the fact that human fatigue is a cumulative process over time, and there are no models available to reflect this phenomenon. Furthermore, the problem of incorrect detection due to facial expression is still not well addressed. In this article, a model based on BP neural network and time cumulative effect was proposed to solve these problems. Experimental data were used to carry out this work and validate the proposed method. Firstly, the Adaboost algorithm was applied to detect faces, and the Kalman filter algorithm was used to trace the face movement. Then, a cascade regression tree-based method was used to detect the 68 facial landmarks and an improved method combining key points and image processing was adopted to calculate the eye aspect ratio (EAR). After that, a BP neural network model was developed and trained by selecting three characteristics: the longest period of continuous eye closure, number of yawns, and percentage of eye closure time (PERCLOS), and then the detection results without and with facial expressions were discussed and analyzed. Finally, by introducing the Sigmoid function, a fatigue detection model considering the time accumulation effect was established, and the drivers’ fatigue state was identified segment by segment through the recorded video. Compared with the traditional BP neural network model, the detection accuracies of the proposed model without and with facial expressions increased by 3.3% and 8.4%, respectively. The number of incorrect detections in the awake state also decreased obviously. The experimental results show that the proposed model can effectively filter out incorrect detections caused by facial expressions and truly reflect that driver fatigue is a time accumulating process.

Scaling effectivity in manifold methodologies to detect driver’s fatigueness and drowsiness state

“…Arefnezhad et al [33] have presented a study of risk of fatigueness during automated driving and used difference of PERCLOS for analyzing the proportion of eye closure in order to determine driver's fatigueness. Adoption of eye closure is also reported in work of Dzuida et al [34] and Shang et al [35] while similar trend of work is also reported by Chen et al [36] where facial detection is initially carried out using Adaboost while tracing of facial movement is done by Kalman filter. Further facial landmarks were detected using regression tree of cascaded form followed by using backpropagation neural network for training.…”

A Multimodal Feature Fusion Framework for Sleep-Deprived Fatigue Detection to Prevent Accidents

“…The accuracy of the proposed model with facial expressions increased by 8.4%. The proposed model can filter out artifacts caused due to facial expressions while detecting driver fatigue [33]. Furthermore, Li, Y. et al proposed a lightweight wearable device based upon a convolution neural network for detecting driver fatigue through eye images [34].…”

Study of fatigue driving based on face and physiological state

“…When these features appeared, the driver may enter a state of mild or severe fatigue. The experiments were measured simultaneously to find the relationship between the facial and physiological states over time [12]. Since the human head may oscillate while driving due to observation of the rearview mirror and reversing mirror, the method of fixing the camera may cause the measurement to be missed.…”