Real-time detection method of driver fatigue state based on deep learning of face video

Cui, Zhe; Sun, Hong-Mei; Yin, Ruo-Nan; Gao, Li; Sun, Hai‐Wei; Jia, Rui-Sheng

doi:10.1007/s11042-021-10930-z

Cited by 18 publications

(8 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The percentage of eye closure time (PERCLOS), eye closure duration (ECD), and frequency of eye closure (FEC) were selected as eye closure-associated fatigue indicators. In the paper of Cui et al [17], a lightweight neural network model was designed to solve the problem of insufficient memory and limited computing power of the current vehiclemounted embedded device. The driver's PERCLOS and frequency of open mouth (FOM) were used to realize the judgment of the driver's fatigue state.…”

Section: Detection Based On Driver Informationmentioning

confidence: 99%

See 1 more Smart Citation

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Chen

Yan

Zhu

et al. 2022

Sensors

View full text Add to dashboard Cite

Fatigue driving has always received a lot of attention, but few studies have focused on the fact that human fatigue is a cumulative process over time, and there are no models available to reflect this phenomenon. Furthermore, the problem of incorrect detection due to facial expression is still not well addressed. In this article, a model based on BP neural network and time cumulative effect was proposed to solve these problems. Experimental data were used to carry out this work and validate the proposed method. Firstly, the Adaboost algorithm was applied to detect faces, and the Kalman filter algorithm was used to trace the face movement. Then, a cascade regression tree-based method was used to detect the 68 facial landmarks and an improved method combining key points and image processing was adopted to calculate the eye aspect ratio (EAR). After that, a BP neural network model was developed and trained by selecting three characteristics: the longest period of continuous eye closure, number of yawns, and percentage of eye closure time (PERCLOS), and then the detection results without and with facial expressions were discussed and analyzed. Finally, by introducing the Sigmoid function, a fatigue detection model considering the time accumulation effect was established, and the drivers’ fatigue state was identified segment by segment through the recorded video. Compared with the traditional BP neural network model, the detection accuracies of the proposed model without and with facial expressions increased by 3.3% and 8.4%, respectively. The number of incorrect detections in the awake state also decreased obviously. The experimental results show that the proposed model can effectively filter out incorrect detections caused by facial expressions and truly reflect that driver fatigue is a time accumulating process.

show abstract

Section: Detection Based On Driver Informationmentioning

confidence: 99%

“…Generally, when a person is in a state of fatigue, the behavior of yawning will increase. Usually, the mouth aspect ratio (MAR) or the frequency of open mouth (FOM) [17] is used to measure the degree of opening and closing of the mouth. Mouth feature points M 1 ~M6 are shown in Figure 3.…”

Section: Definition Of Mouth Aspect Ratiomentioning

confidence: 99%

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Chen

Yan

Zhu

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…In order to prevent the single-dimensional information incompleteness caused by emotion detection in the evaluation of students' learning state, this paper uses fatigue detection as a dimension of learning state evaluation, and again uses convolutional neural network to build a human eye state recognition model to detect student fatigue [26,27], classify images with eyes open and eyes closed. By recording the number of closed eyes of students in a period of time to evaluate the degree of student fatigue.…”

Section: Fatigue Testingmentioning

confidence: 99%

An Analysis Model of Learners’ Online Learning Status Based on Deep Neural Network and Multi-Dimensional Information Fusion

Ma¹,

Tang²,

Ma³

et al. 2023

Computer Modeling in Engineering &Amp; Sciences

View full text Add to dashboard Cite

The learning status of learners directly affects the quality of learning. Compared with offline teachers, it is difficult for online teachers to capture the learning status of students in the whole class, and it is even more difficult to continue to pay attention to students while teaching. Therefore, this paper proposes an online learning state analysis model based on a convolutional neural network and multi-dimensional information fusion. Specifically, a facial expression recognition model and an eye state recognition model are constructed to detect students' emotions and fatigue, respectively. By integrating the detected data with the homework test score data after online learning, an analysis model of students' online learning status is constructed. According to the PAD model, the learning state is expressed as three dimensions of students' understanding, engagement and interest, and then analyzed from multiple perspectives. Finally, the proposed model is applied to actual teaching, and procedural analysis of 5 different types of online classroom learners is carried out, and the validity of the model is verified by comparing with the results of the manual analysis.

show abstract

“…Although not always easy to capture, some of the affective states brought out by experts during our workshops can be automatically detected using machine learning. Deep learning approaches applied to driver-facing footage, for instance, have shown promising results in automatically identifying some of the affective states, such as, distracted or attentive driving [43], [44], different types of human emotions [45], [46], and tired or energetic [47], [48]. Alternatively, calm or aggressive driving is accurately detected using telematics incident data [2], [3], while more complex affective states such as confidence and insecurity are still difficult to detect.…”

Section: B Effects Of Contextual Factors On Hgv Drivers' Performancementioning

confidence: 99%

Contextual Intelligent Decisions: Expert Moderation of Machine Outputs for Fair Assessment of Commercial Driving

Mase¹,

Pekaslan²,

Agrawal³

et al. 2022

Preprint

View full text Add to dashboard Cite

Commercial driving is a complex multifaceted task influenced by personal traits and external contextual factors, such as weather, traffic, road conditions, etc. Previous intelligent commercial driver-assessment systems do not consider these factors when analysing the impact of driving behaviours on road safety, potentially producing biased, inaccurate, and unfair assessments. In this paper, we introduce a methodology (Expertcentered Driver Assessment) towards a fairer automatic road safety assessment of drivers' behaviours, taking into consideration behaviours as a response to contextual factors. The contextual moderation embedded within the intelligent decisionmaking process is underpinned by expert input, comprising of a range of associated stakeholders in the industry. Guided by the literature and expert input, we identify critical factors affecting driving and develop an interval-valued response-format questionnaire to capture the uncertainty of the influence of factors and variance amongst experts' views. Questionnaire data are modelled and analysed using fuzzy sets, as they provide a suitable computational approach to be incorporated into decisionmaking systems with uncertainty. The methodology has allowed us to identify the factors that need to be considered when moderating driver sensor data, and to effectively capture experts' opinions about the effects of the factors. An example of our methodology using Heavy Goods Vehicles professionals input is provided to demonstrate how the expert-centred moderation can be embedded in intelligent driver assessment systems.

show abstract

Real-time detection method of driver fatigue state based on deep learning of face video

Cited by 18 publications

References 42 publications

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

An Analysis Model of Learners’ Online Learning Status Based on Deep Neural Network and Multi-Dimensional Information Fusion

Contextual Intelligent Decisions: Expert Moderation of Machine Outputs for Fair Assessment of Commercial Driving

Contact Info

Product

Resources

About