Unusual Driver Behavior Detection in Videos Using Deep Learning Models

Abstract

Anomalous driving behavior detection is becoming more popular since it is vital in ensuring the safety of drivers and passengers in vehicles. Road accidents happen for various reasons, including health, mental stress, and fatigue. It is critical to monitor abnormal driving behaviors in real time to improve driving safety, raise driver awareness of their driving patterns, and minimize future road accidents. Many symptoms appear to show this condition in the driver, such as facial expressions or abnormal actions. The abnormal activity was among the most common causes of road accidents, accounting for nearly 20% of all accidents, according to international data on accident causes. To avoid serious consequences, abnormal driving behaviors must be identified and avoided. As it is difficult to monitor anyone continuously, automated detection of this condition is more effective and quicker. To increase drivers’ recognition of their driving behaviors and prevent potential accidents, a precise monitoring approach that detects abnormal driving behaviors and identifies abnormal driving behaviors is required. The most common activities performed by the driver while driving is drinking, eating, smoking, and calling. These types of driver activities are considered in this work, along with normal driving. This study proposed deep learning-based detection models for recognizing abnormal driver actions. This system is trained and tested using a newly created dataset, including five classes. The main classes include Driver-smoking, Driver-eating, Driver-drinking, Driver-calling, and Driver-normal. For the analysis of results, pre-trained and fine-tuned CNN models are considered. The proposed CNN-based model and pre-trained models ResNet101, VGG-16, VGG-19, and Inception-v3 are used. The results are compared by using the performance measures. The results are obtained 89%, 93%, 93%, 94% for pre-trained models and 95% by using the proposed CNN-based model. Our analysis and results revealed that our proposed CNN base model performed well and could effectively classify the driver’s abnormal behavior.

Comprehensive Assessment of Artificial Intelligence Tools for Driver Monitoring and Analyzing Safety Critical Events in Vehicles

“…Later, Yan et al [ 64 ] used the SEU dataset to classify six driver behaviors (responding to phone call, eating while driving, operating the shift gear, correct driving position with hands on wheel, playing with phone while driving, and driving while smoking) via a Gaussian Mixture Model to extract skin-like regions and using CNN to generate action labels on videos, achieving a mean average precision (mAP) of 97.97%. Abosaq et al [ 65 ] proposed a customized CNN model ( Figure 5 ) to recognize normal and abnormal driver actions (including driver smoking, driver eating, driver drinking, driver calling, and driver normal) from driver videos, and achieved 95% accuracy on the prepared testing dataset. Yang et al [ 66 ] investigated the impacts of feature selection on driver cognitive distraction detection and validation in real-world non-automated and Level 2 automated driving scenarios.…”

Abnormal Behavior Detection in Video Surveillance Using Inception-v3 Transfer Learning Approaches

“…The framework demonstrated an accuracy of 88% on undistorted videos and 86% on distorted videos. [4] presented a deep learning system to identify abnormal behavior in drivers, using a new dataset that includes categories such as smoking, eating, drinking, calling, and normal behavior. The study employs pre-trained and fine-tuned CNN models, such as ResNet101, VGG-16, VGG-19, and a proposed CNN model, to analyze the results.…”

“…Computer vision and machine learning methods are utilized to detect and identify human activities. The use of diverse video characteristics enables an understanding of human behavior and actions, which, in turn, aids in categorizing activities as either normal or abnormal [4]. However, the recognition of human activity poses challenges due to the complexity of human behavior, making it difficult to differentiate between normal and abnormal activities [5], [6].…”