DriverMVT: In-Cabin Dataset for Driver Monitoring including Video and Vehicle Telemetry Information

Othman, Walaa; Kashevnik, Alexey; Ali, Ammar; Shilov, Nikolay

doi:10.3390/data7050062

Cited by 13 publications

(15 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It can be observed from the table results that YOLOv6_1 outperformed the other detection models on the DriverMVT dataset. Moreover, the fine-tuned algorithms were evaluated on the DriverMVT (Driver Monitoring with Videos and Telemetry) dataset [3] to generalize the model. However, this dataset presents driver behavior inside a vehicle, like drowsiness and an unbuttoned belt.…”

Section: Simulation Resultsmentioning

confidence: 99%

Visual-Based Children and Pet Rescue from Suffocation and Incidence of Hyperthermia Death in Enclosed Vehicles

Moussa

Shoitan

Cho

et al. 2023

Sensors

View full text Add to dashboard Cite

Over the past several years, many children have died from suffocation due to being left inside a closed vehicle on a sunny day. Various vehicle manufacturers have proposed a variety of technologies to locate an unattended child in a vehicle, including pressure sensors, passive infrared motion sensors, temperature sensors, and microwave sensors. However, these methods have not yet reliably located forgotten children in the vehicle. Recently, visual-based methods have taken the attention of manufacturers after the emergence of deep learning technology. However, the existing methods focus only on the forgotten child and neglect a forgotten pet. Furthermore, their systems only detect the presence of a child in the car with or without their parents. Therefore, this research introduces a visual-based framework to reduce hyperthermia deaths in enclosed vehicles. This visual-based system detects objects inside a vehicle; if the child or pet are without an adult, a notification is sent to the parents. First, a dataset is constructed for vehicle interiors containing children, pets, and adults. The proposed dataset is collected from different online sources, considering varying illumination, skin color, pet type, clothes, and car brands for guaranteed model robustness. Second, blurring, sharpening, brightness, contrast, noise, perspective transform, and fog effect augmentation algorithms are applied to these images to increase the training data. The augmented images are annotated with three classes: child, pet, and adult. This research concentrates on fine-tuning different state-of-the-art real-time detection models to detect objects inside the vehicle: NanoDet, YOLOv6_1, YOLOv6_3, and YOLO7. The simulation results demonstrate that YOLOv6_1 presents significant values with 96% recall, 95% precision, and 95% F1.

show abstract

Section: Simulation Resultsmentioning

confidence: 99%

Visual-Based Children and Pet Rescue from Suffocation and Incidence of Hyperthermia Death in Enclosed Vehicles

Moussa

Shoitan

Cho

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Pé rez-Jimé nez et al [23] proposes a system based on analysis of visual images and combines the results of different types of classifiers to achieve robust and real-time detection. Othman et al [24] have presented a driver monitoring system using in-cabin monitoring and deep learning. They have gathered a video and telemetry dataset for driver monitoring.…”

Section: Vision Methods For In Cabin Occupant Monitoringmentioning

confidence: 99%

“…This dataset consists of synthetic and real annotated images for monitoring front seats. The Adience benchmark [24] is used to train and evaluate the Muti-task CNN model. this dataset is constructed for age and gender recognition.…”

Section: Datasetsmentioning

confidence: 99%

In-cabin occupant monitoring system based on improved Yolo, deep reinforcement learning, and multi-task CNN for autonomous driving

KHRAIEF OULED AZAIZ,

DACLEU NDENGUE

2023

Fifteenth International Conference on Machine Vision (ICMV 2022)

View full text Add to dashboard Cite

Vehicle in-cabin occupant monitoring system is becoming a crucial feature of the automobile industry and challenging research topic to enhance both vehicle safety, security, and comfort of conventional and future intelligent vehicles. Precise information about the number, position, and characteristics of occupants as well as objects located inside the vehicle must be available. Current industrial systems for seat occupancy detection are based on multiple weight sensors, capacitive sensors, electric field, or ultrasonic sensors. They cannot necessarily make the right distinction in borderline cases. A simple pressure sensor cannot tell whether the weight on the seat comes from a person or an inanimate object. Recently, the Artificial Intelligence (AI) based advanced systems have attracted attention for various fields such as automobile industry. Especially, with the advancement of deep learning that has shown very high classification accuracies compared to hand-crafted features on various computer vision tasks. For the above reasons, we propose a new automatic AI occupant monitoring system based on two cameras installed inside the vehicle. Our goal is to develop an automatic detection and recognition system with high accuracy performance, low computational cost and small weight model. Our system fuses our modified deep convolutional network Yolo model and deep reinforcement learning to detect and classify passengers and objects inside the vehicle. It can predict the gender, the age and the emotion of occupants based on our proposed muti task convolutional neural networks. In our end-to-end system, this approach is more efficient time and memory wise by solving all the tasks in the same process and storing a single CNN instead of storing a CNN for each task. Principal applications of our system are intelligent airbag management, seat belt reminder, life presence and in shared cabin preferences. We perform comparative evaluation based on the public datasets SVIRO, TiCaM, Aff-Wild and Adience dataset to demonstrate the superior performance of our proposed system.

show abstract

“…Pappalardo et al have applied the decision tree method to analyze the lane support system in [38]. Othman et al have presented a driver monitoring system using in-cabin monitoring and deep learning [39]. They have gathered a video and telemetry dataset for driver monitoring.…”

Section: Related Workmentioning

confidence: 99%

In-Cabin Monitoring System for Autonomous Vehicles

Mishra

Lee

Kim

et al. 2022

Sensors

View full text Add to dashboard Cite

In this paper, we have demonstrated a robust in-cabin monitoring system (IMS) for safety, security, surveillance, and monitoring, including privacy concerns for personal and shared autonomous vehicles (AVs). It consists of a set of monitoring cameras and an onboard device (OBD) equipped with artificial intelligence (AI). Hereafter, this combination of a camera and an OBD is referred to as the AI camera. We have investigated the issues for mobility services in higher levels of autonomous driving, what needs to be monitored, how to monitor, etc. Our proposed IMS is an on-device AI system that indigenously has improved the privacy of the users. Furthermore, we have enlisted the essential actions to be considered in an IMS and developed an appropriate database (DB). Our DB consists of multifaced scenarios important for monitoring the in-cabin of the higher-level AVs. Moreover, we have compared popular AI models applied for object and occupant recognition. In addition, our DB is available on request to support the research on the development of seamless monitoring of the in-cabin higher levels of autonomous driving for the assurance of safety and security.

show abstract

DriverMVT: In-Cabin Dataset for Driver Monitoring including Video and Vehicle Telemetry Information

Cited by 13 publications

References 22 publications

Visual-Based Children and Pet Rescue from Suffocation and Incidence of Hyperthermia Death in Enclosed Vehicles

Visual-Based Children and Pet Rescue from Suffocation and Incidence of Hyperthermia Death in Enclosed Vehicles

In-cabin occupant monitoring system based on improved Yolo, deep reinforcement learning, and multi-task CNN for autonomous driving

In-Cabin Monitoring System for Autonomous Vehicles

Contact Info

Product

Resources

About