Real Time Driver Fatigue Detection System Based on Multi-Task ConNN

Savaş, Burcu Kır; Becerikli, Yaşar

doi:10.1109/access.2020.2963960

Cited by 126 publications

(44 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e features of the eyes and mouth were used to model the behavior of the driver. e changes in these characteristics were used to monitor the driver's fatigue [16]. Liu et al proposed a fatigue detection algorithm based on the deep learning facial expression analysis.…”

Section: Related Workmentioning

confidence: 99%

Driver Fatigue Detection Based on Facial Key Points and LSTM

Chen

Xin

Liu

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

In recent years, fatigue driving has been a serious threat to the traffic safety, which makes the research of fatigue detection a hotspot field. Research on fatigue recognition has a great significance to improve the traffic safety. However, the existing fatigue detection methods still have room for improvement in detection accuracy and efficiency. In order to detect whether the driver has fatigue driving, this paper proposes a fatigue state recognition algorithm. The method first uses MTCNN (multitask convolutional neural network) to detect human face, and then DLIB (an open-source software library) is used to locate facial key points to extract the fatigue feature vector of each frame. The fatigue feature vectors of multiple frames are spliced into a temporal feature sequence and sent to the LSTM (long short-term memory) network to obtain a final fatigue feature value. Experiments show that compared with other methods, the fatigue state recognition algorithm proposed in this paper has achieved better results in accuracy. The average accuracy of the proposed method in detecting key points of the face is as high as 93%, and the running time is less than half of the ordinary DLIB method.

show abstract

Section: Related Workmentioning

confidence: 99%

Driver Fatigue Detection Based on Facial Key Points and LSTM

Chen

Xin

Liu

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Bergasa et al [32] calculated the following six parameters: PERCLOS, duration of closed eyes, blink frequency, nod frequency, facial position and fixed gaze, and combined these parameters with a fuzzy classifier to infer driver fatigue. Savaş et.al [51] proposed a Multi-tasking Convolutional Neural Network (ConNN) model to detect driver drowsiness/fatigue. Both mouth and eye information are classified into a single model at the same time.…”

Section: Fatigue Detection Based On Mixed Behaviour Characteristicsmentioning

confidence: 99%

Real time detection of driver fatigue based on CNN‐LSTM

Liu

et al. 2021

IET Image Processing

View full text Add to dashboard Cite

Fatigue driving is one of the main causes of traffic accidents. In order to solve this problem, a new Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM) based real-time driver fatigue detection method is proposed. First of all, using simple linear clustering algorithm (SLIC), the driver's image is divided into super pixels of uniform size, which are used as input of CNN, and CNN is trained to automatically learn the features of eyes and mouth contained in the image, and then the location and area of eyes and mouth are obtained by using the trained CNN. On this basis, the eye feature parameter Perclos, mouth feature parameter MClosed and face orientation feature parameter Phdown are extracted, and the above feature parameters on the continuous time series and steering wheel angle feature parameter SA are taken as the input of LSTM, and the fatigue level is taken as the output to detect the fatigue state of the driver in real time. Experimental data shows that this method can not only overcome the influence of illumination, background, angle and individual differences, but also the accuracy of detection can reach 99.78%, and the average detection time is 16.94 ms/frame.

show abstract

“…Similar to PERCLOS, FOM (frequency of mouth) [32] calculates the ratio of mouth open within a certain period and then infers the driver's yawn frequency. FOM is calculated by: There is a SoftMax layer after the last full connection layer.…”

Section: Comprehensive Judgementmentioning

confidence: 99%

ICONet: A Lightweight Network with Greater Environmental Adaptivity

Huang

et al. 2020

Symmetry

View full text Add to dashboard Cite

With the increasing popularity of artificial intelligence, deep learning has been applied to various fields, especially in computer vision. Since artificial intelligence is migrating from cloud to edge, deep learning nowadays should be edge-oriented and adaptive to complex environments. Aiming at these goals, this paper proposes an ICONet (illumination condition optimized network). Based on OTSU segmentation algorithm and fuzzy c-means clustering algorithm, the illumination condition classification subnet increases the environmental adaptivity of our network. The reduced time complexity and optimized size of our convolutional neural network (CNN) model enables the implementation of ICONet on edge devices. In the field of fatigue driving, we test the performance of ICONet on YawDD and self-collected datasets. Our network achieves a general accuracy of 98.56% and our models are about 590 kilobytes. Compared to other proposed networks, the ICONet shows significant success and superiority. Applying ICONet to fatigue driving detection is helpful to solve the symmetry of the needs of edge-oriented detection under complex illumination condition environments and the scarcity of related approaches.

show abstract

Real Time Driver Fatigue Detection System Based on Multi-Task ConNN

Cited by 126 publications

References 25 publications

Driver Fatigue Detection Based on Facial Key Points and LSTM

Driver Fatigue Detection Based on Facial Key Points and LSTM

Real time detection of driver fatigue based on CNN‐LSTM

ICONet: A Lightweight Network with Greater Environmental Adaptivity

Contact Info

Product

Resources

About