Toward Extremely Lightweight Distracted Driver Recognition With Distillation-Based Neural Architecture Search and Knowledge Transfer

Abstract

The number of traffic accidents has been continuously increasing in recent years worldwide. Many accidents are caused by distracted drivers, who take their attention away from driving. Motivated by the success of Convolutional Neural Networks (CNNs) in computer vision, many researchers developed CNN-based algorithms to recognize distracted driving from a dashcam and warn the driver against unsafe behaviors. However, current models have too many parameters, which is unfeasible for vehicle-mounted computing. This work proposes a novel knowledge-distillation-based framework to solve this problem. The proposed framework first constructs a high-performance teacher network by progressively strengthening the robustness to illumination changes from shallow to deep layers of a CNN. Then, the teacher network is used to guide the architecture searching process of a student network through knowledge distillation. After that, we use the teacher network again to transfer knowledge to the student network by knowledge distillation. Experimental results on the Statefarm Distracted Driver Detection Dataset and AUC Distracted Driver Dataset show that the proposed approach is highly effective for recognizing distracted driving behaviors from photos: (i) the teacher network's accuracy surpasses the previous best accuracy; (ii) the student network achieves very high accuracy with only 0.42M parameters (around 55% of the previous most lightweight model). Furthermore, the student network architecture can be extended to a spatial-temporal 3D CNN for recognizing distracted driving from video clips. The 3D student network largely surpasses the previous best accuracy with only 2.03M parameters on the Drive&Act Dataset.

A novel multi-sensor hybrid fusion framework

“…Lightweight CNN has received increasing attention due to fewer weighting parameters and lower computational requirements. The key points to achieve CNN lightweight are to reduce the number of parameters and computational complexity of the model, and the common methods include using depthwise separable convolution instead of regular convolution [44], model pruning [45], and knowledge distillation [46,47]. Model pruning usually requires fine-tuning the model, which is relatively complex and sometimes even requires re-training the entire model.…”

Driver Distraction Behavior Detection Framework Based on the DWPose Model, Kalman Filtering, and Multi-Transformer

“…In other studies, the number of model parameters has been reduced by optimizing the structure of deep learning models [26,27]. As knowledge distillation has become an important lightweighting tool for deep learning models, it has been applied to the field of driver distraction monitoring [28,29]. Liu, et al [29] reduced the number of parameters of a deep learning model to 0.42M based on the knowledge distillation method, which facilitated the in-vehicle application of the driving behavior detection model.…”

“…As knowledge distillation has become an important lightweighting tool for deep learning models, it has been applied to the field of driver distraction monitoring [28,29]. Liu, et al [29] reduced the number of parameters of a deep learning model to 0.42M based on the knowledge distillation method, which facilitated the in-vehicle application of the driving behavior detection model. However, an important drawback of knowledge distillation is the requirement of a very large training set for the teacher model to extract useful knowledge to guide the student model.…”

Resource-Conscious Architecture Search for Driver Distraction Detection: A Pareto Optimised Multi-Objective Approach

“…al [48] proposed a fine-grained detection of driver distraction, incorporating multisource data, but their NAS-generated CNN has a large volume (2.7M Params) and does not adequately cater to the specific needs of DDD. Fourth, Liu et al [49] introduced a teacherstudent model with knowledge distillation for the same task. As far as we know, this work is the sole endeavor in using benchmark public datasets and NAS for DDD, making it the most relevant point of comparison for our research.…”