Real-time facial state recognition and fatigue analysis based on deep neural networks

Abstract

Aiming at the problems such as large parameter count of facial state recognition model in driver fatigue detection which is difficult to be deployed, low accuracy, slow speed, etc., a lightweight real-time facial state recognition model YOLOv5-fatigue based on YOLOv5n is proposed; Firstly, a bilateral convolution (BConv) is proposed, which can fully utilize the feature information in the channel; Then an innovative deep convolution module (DBS) is proposed, which utilizes the module to reduce the number of network parameters as well as the amount of computation; Lastly, the NAM attention mechanism is added to solve the problem of accuracy degradation due to the lightweighting of the model; In this paper, we first recognize the facial state by YOLOv5-fatigue, and then use the proportion of eyes closed per unit of time (PERCLOS) and the proportion of mouth closed per unit of time (POM) to determine fatigue. Experiments on the self-built VIGP-fatigue dataset show that the AP50 of the proposed method is improved to 92.6%, the inference time is reduced to 2.1ms,and the amount of parameters is reduced to 1.01M.By comparing with the driving video in real situation, it is found that the accuracy of fatigue detection reaches 94.7%.