Multimodal Features for Detection of Driver Stress and Fatigue: Review

Němcová, Andrea; Svozilová, Veronika; Bucsuházy, Kateřina; Smíšek, Radovan; Mézl, Martin; Hesko, Branislav; Belák, Michal; Bilík, Martin; Maxera, Pavel; Seitl, Martin; Dominik, Tomáš; Semela, Marek; Šucha, Matúš; Kolář, Radim

doi:10.1109/tits.2020.2977762

Cited by 69 publications

(28 citation statements)

References 153 publications

(323 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent studies, the Multimodal-based (M-DFD) systems, using deep learning architecture, played a vital role in recognizing the driver’s different activities and fatigue at different levels. Nowadays, many authors use distinct data types [ 189 , 190 , 191 , 192 , 193 ], such as the physical conditions of the driver, audio, visual features, and car information; the main data sources are the images of the driver, which include the face, arms, and hands, taken with a camera placed inside the car. Several authors developed a way to integrate sensor data into the vision-based distracted driver detection model, to improve the generalization ability of the system.…”

Section: Architectural Comparisonsmentioning

confidence: 99%

Driver Fatigue Detection Systems Using Multi-Sensors, Smartphone, and Cloud-Based Computing Platforms: A Comparative Analysis

Abbas

Alsheddy

2020

Sensors

View full text Add to dashboard Cite

Internet of things (IoT) cloud-based applications deliver advanced solutions for smart cities to decrease traffic accidents caused by driver fatigue while driving on the road. Environmental conditions or driver behavior can ultimately lead to serious roadside accidents. In recent years, the authors have developed many low-cost, computerized, driver fatigue detection systems (DFDs) to help drivers, by using multi-sensors, and mobile and cloud-based computing architecture. To promote safe driving, these are the most current emerging platforms that were introduced in the past. In this paper, we reviewed state-of-the-art approaches for predicting unsafe driving styles using three common IoT-based architectures. The novelty of this article is to show major differences among multi-sensors, smartphone-based, and cloud-based architectures in multimodal feature processing. We discussed all of the problems that machine learning techniques faced in recent years, particularly the deep learning (DL) model, to predict driver hypovigilance, especially in terms of these three IoT-based architectures. Moreover, we performed state-of-the-art comparisons by using driving simulators to incorporate multimodal features of the driver. We also mention online data sources in this article to test and train network architecture in the field of DFDs on public available multimodal datasets. These comparisons assist other authors to continue future research in this domain. To evaluate the performance, we mention the major problems in these three architectures to help researchers use the best IoT-based architecture for detecting DFDs in a real-time environment. Moreover, the important factors of Multi-Access Edge Computing (MEC) and 5th generation (5G) networks are analyzed in the context of deep learning architecture to improve the response time of DFD systems. Lastly, it is concluded that there is a research gap when it comes to implementing the DFD systems on MEC and 5G technologies by using multimodal features and DL architecture.

show abstract

Section: Architectural Comparisonsmentioning

confidence: 99%

Driver Fatigue Detection Systems Using Multi-Sensors, Smartphone, and Cloud-Based Computing Platforms: A Comparative Analysis

Abbas

Alsheddy

2020

Sensors

View full text Add to dashboard Cite

show abstract

“…However, this method uses relatively large equipment, which is not applicable for wearable applications in human activities such as working and exercising. [ 9,14 ]…”

Section: Introductionmentioning

confidence: 99%

Wireless and Flexible Skin Moisture and Temperature Sensor Sheets toward the Study of Thermoregulator Center

Fujita

Honda

et al. 2021

Adv Healthcare Materials

View full text Add to dashboard Cite

A disorder in the thermoregulator center in a human body leads to some potential diseases such as fever and hyperthyroidism. To predict these diseases early, monitoring the health condition of the human body due to the influence of thermoregulation disorders is important. Although extensive works are performed on sweat‐rate detection by constructing microfluidic channels, skin‐moisture evaporation before sweating remains unknown. This work proposes a wireless and flexible sensor sheet to investigate the thermoregulatory responses of different people under cold stimulation and exercise by measuring the temperature and moisture variations on the finger skin. An integrated flexible sensor system consists of a ZnIn2S4 nanosheet‐based humidity sensor and carbon nanotube/SnO2 temperature sensor. The results exhibit distinct thermoregulation abilities of five volunteers. Interestingly, the sudden increase in finger moisture that results from the excitation by the sympathetic nerve is observed during the cold‐stimulus test. Although further studies are required to predict the potential diseases resulted from thermoregulation disorders in human body, this study provides a possibility of continuous and real‐time monitoring of thermoregulatory activities via skin moisture and temperature detection using a flexible sensor sheet.

show abstract

“…e sensitivity of nonintrusive measurements to drowsiness depends on the time window setting method and calculation parameters of measurements, which greatly determines the accuracy of drowsiness-detection [18,19]. Many studies concerning measurements and detection methods of drowsy driving have been conducted, which can be found in some review literature [11,12,14,[20][21][22]. is paper focuses on the optimization of calculation parameters of nonintrusive drowsiness measurements.…”

Section: Introductionmentioning

confidence: 99%

Extraction of Optimal Measurements for Drowsy Driving Detection considering Driver Fingerprinting Differences

Zhang

et al. 2021

Journal of Advanced Transportation

View full text Add to dashboard Cite

Contributions of measurements for detecting drowsy driving are determined by calculation parameters, which are directly related to the accuracy of drowsiness detection. The previous studies utilized the same Unified Calculation Parameters (UCPs) to compute each driver’s measurements. However, since each driver has unique driving behavior characteristics, namely, driver fingerprinting, Individual Drivers’ Best Calculation Parameters (IDBCPs) making measurements more discriminative for drowsiness are various. Regardless of the difference in driver fingerprinting among the drivers being tested, using UCPs instead of IDBCPs to compute measurements will limit the drowsiness-detection performance of the measurements and reduce drowsiness-detection accuracies at the individual driver level. Thus, this paper proposed a model to optimize calculation parameters of individual driver’s measurements and to extract individual driver’s measurements that effectively distinguish drowsy driving. Through real vehicle experiments, we collected naturalistic driving data and subjective drowsy levels evaluated by the Karolinska Sleepiness Scale. Eight nonintrusive drowsiness-related measurements were calculated by double-layer sliding time windows. In the proposed model, we firstly applied the Wilcoxon test to analyze differences between measurements of the awake state and drowsy state, and constructed the fitness function reflecting the relationship between the calculation parameters and measurement’s drowsiness-detection performance. Secondly, the genetic algorithms were used to optimize fitness functions to obtain measured IDBCPs. Finally, we selected measurements calculated by IDBCPs that can distinguish drowsy driving to constitute individual drivers’ optimal drowsiness-detection measurement set. To verify the advantages of IDBCPs, the measurements calculated by UCPs and IDBCPs were, respectively, used to build driver-specific drowsiness-detection models: DF_U and DF_I based on the Fisher discriminant algorithm. The mean drowsiness-detection accuracies of DF_U and DF_I were, respectively, 85.25% and 91.06%. It indicated that IDBCPs could enhance measurements’ drowsiness-detection performance and improve the drowsiness-detection accuracies. This paper contributed to the establishment of personalized drowsiness-detection models considering driver fingerprinting differences.

show abstract

Multimodal Features for Detection of Driver Stress and Fatigue: Review

Cited by 69 publications

References 153 publications

Driver Fatigue Detection Systems Using Multi-Sensors, Smartphone, and Cloud-Based Computing Platforms: A Comparative Analysis

Driver Fatigue Detection Systems Using Multi-Sensors, Smartphone, and Cloud-Based Computing Platforms: A Comparative Analysis

Wireless and Flexible Skin Moisture and Temperature Sensor Sheets toward the Study of Thermoregulator Center

Extraction of Optimal Measurements for Drowsy Driving Detection considering Driver Fingerprinting Differences

Contact Info

Product

Resources

About