Long Short-term Memory Network Based Fatigue Detection with Sequential Mouth Feature

Fei, Yanling; Li, Bin; Wang, Heng; Tian, Lianfang

doi:10.1109/isas49493.2020.9378877

Cited by 8 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It enhances detection feature stability and achieves a precision of 94.44% by employing intermediate frame filtering to prevent duplicate detections. To better accommodate individual differences and enhance the model's generalization ability, Fei et al [5] proposed a fatigue detection system based on sequential mouth features with a long short-term memory (LSTM) network. To mitigate the impact of lighting conditions and facial expression changes, Histogram of Oriented Gradient (HOG) features and a Support Vector Machine (SVM) classifier are utilized for face detection.…”

Section: Single-feature Detection Methodsmentioning

confidence: 99%

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

Deng,

Yang,

Guo

et al. 2024

Preprint

View full text Add to dashboard Cite

Fatigue driving stands as a primary contributor to severe traffic accidents. Despite notable advancements in fatigue monitoring techniques, current methods frequently overlook the utilization of shifts in drivers' body postures induced by fatigue. These postural changes can offer valuable indicators of drivers' levels of alertness and degree of fatigue. Therefore, a real-time system with the ability to process body information is needed. This study presents a novel framework that integrates facial fatigue characteristics with alterations in drivers' upper body postures to comprehensively monitor fatigue across multiple dimensions. The neural network architecture at the core of the proposed approach utilizes a video vision transformer framework, with the self-attention mechanism optimized to accommodate the magnitude of fatigue-induced posture changes. This optimization ensures high accuracy while simultaneously reducing computational expenses. This gives the approach the potential for embedded deployment. According to requirements, the standard DMD dataset captured in real-world driving scenarios has been selected for validating the proposed method. In terms of accuracy, compared to detection methods that solely rely on facial and upper body poses, the multi-feature fatigue detection approach has achieved an improvement of 3.4% and 11.9%, reaching 96.24%. Additionally, it has been observed that the adoption of the Dynamic Distributed Attention-based backbone has led to a significant increase in inference speed of over 30%.

show abstract

Section: Single-feature Detection Methodsmentioning

confidence: 99%

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

Deng,

Yang,

Guo

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The CNN extracts spatial features from individual frames, and the LSTM network analyzes the temporal features of driver actions between the adjacent frames. For example, some authors attempted to employ this CNN-LSTM network to build a DDD system depending only on the analysis of yawning features [19][20][21]. Most of them used a public dataset named YawDD.…”

Section: Spatiotemporal-based Systemsmentioning

confidence: 99%

“…One of the main differences between these studies is the type of inputs to the models. Xie et al [19] use the whole frames as inputs, Zhang et al [20] use frame edges, and Fei et al [21] focus on the extracted mouth regions from the drivers' faces. Other researchers utilized the CNN-LSTM network to learn the spatiotemporal feature of drivers' eyes.…”

Section: Spatiotemporal-based Systemsmentioning

confidence: 99%

“…The different style of drowsiness actions for each driver is the reason behind the complexity of DDD systems. For that, building a system that depends on only yawning detection [13,[19][20][21]23] or eye state recognition [15][16][17]22] cannot be used as an effective real-life DDD system.…”

Section: Nthu-dddmentioning

confidence: 99%

“…Several studies crop the mouth region from the face driver to analyze the yawning frequency [21], where yawning symptoms are more complex than a wide-open mouth. It affects many areas in the face as raised eyebrows, closed eyes, and stretched face.…”

Section: Nthu-dddmentioning

confidence: 99%

See 2 more Smart Citations

A Lightweight Driver Drowsiness Detection System Using 3DCNN With LSTM

Alameen¹,

Alhothali²

2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

Today, fatalities, physical injuries, and significant economic losses occur due to car accidents. Among the leading causes of car accidents is drowsiness behind the wheel, which can affect any driver. Drowsiness and sleepiness often have associated indicators that researchers can use to identify and promptly warn drowsy drivers to avoid potential accidents. This paper proposes a spatiotemporal model for monitoring drowsiness visual indicators from videos. This model depends on integrating a 3D convolutional neural network (3D-CNN) and long short-term memory (LSTM). The 3DCNN-LSTM can analyze long sequences by applying the 3D-CNN to extract spatiotemporal features within adjacent frames. The learned features are then used as the input of the LSTM component for modeling high-level temporal features. In addition, we investigate how the training of the proposed model can be affected by changing the position of the batch normalization (BN) layers in the 3D-CNN units. The BN layer is examined in two different placement settings: before the non-linear activation function and after the non-linear activation function. The study was conducted on two publicly available drowsy drivers datasets named 3MDAD and YawDD. 3MDAD is mainly composed of two synchronized datasets recorded from the frontal and side views of the drivers. We show that the position of the BN layers increases the convergence speed and reduces overfitting on one dataset but not the other. As a result, the model achieves a test detection accuracy of 96%, 93%, and 90% on YawDD, Side-3MDAD, and Front-3MDAD, respectively.

show abstract

Intelligent Fatigue Driving Detection Method Based on Fusion of Smartphone and Smartwatch Data

Wang,

Li,

Xing

et al. 2024

Communications in Computer and Information Science

View full text Add to dashboard Cite

Long Short-term Memory Network Based Fatigue Detection with Sequential Mouth Feature

Cited by 8 publications

References 15 publications

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

A Lightweight Driver Drowsiness Detection System Using 3DCNN With LSTM

Intelligent Fatigue Driving Detection Method Based on Fusion of Smartphone and Smartwatch Data

Contact Info

Product

Resources

About