Multi-Timescale Drowsiness Characterization Based on a Video of a Driver’s Face

Abstract

Drowsiness is a major cause of fatal accidents, in particular in transportation. It is therefore crucial to develop automatic, real-time drowsiness characterization systems designed to issue accurate and timely warnings of drowsiness to the driver. In practice, the least intrusive, physiology-based approach is to remotely monitor, via cameras, facial expressions indicative of drowsiness such as slow and long eye closures. Since the system’s decisions are based upon facial expressions in a given time window, there exists a trade-off between accuracy (best achieved with long windows, i.e., at long timescales) and responsiveness (best achieved with short windows, i.e., at short timescales). To deal with this trade-off, we develop a multi-timescale drowsiness characterization system composed of four binary drowsiness classifiers operating at four distinct timescales (5 s, 15 s, 30 s, and 60 s) and trained jointly. We introduce a multi-timescale ground truth of drowsiness, based on the reaction times (RTs) performed during standard Psychomotor Vigilance Tasks (PVTs), that strategically enables our system to characterize drowsiness with diverse trade-offs between accuracy and responsiveness. We evaluated our system on 29 subjects via leave-one-subject-out cross-validation and obtained strong results, i.e., global accuracies of 70%, 85%, 89%, and 94% for the four classifiers operating at increasing timescales, respectively.

Heart Rate Variability for Classification of Alert Versus Sleep Deprived Drivers in Real Road Driving Conditions

“…The KSS values are used as target values when training the classifiers. Alternative approaches used as ground truth in driver sleepiness studies include EEG [24], [25], reaction time tests [26], and expert ratings based on observations [12], [15]. However, reaction time tests are difficult to administer in real-road driving and video-based expert ratings have been found to be unreliable [27].…”

Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal Network

“…Many studies [ 15 , 16 , 17 , 18 ] have been conducted on the state of the driver’s eye based on a video surveillance system to detect the eyes and calculate their frequency of blinking to check the driver’s fatigue level. Some studies [ 18 ], such as Adaboost’s [ 18 ], have used a cascade of classifiers for the rapid detection of the ocular area.…”

“…Some studies [ 18 ], such as Adaboost’s [ 18 ], have used a cascade of classifiers for the rapid detection of the ocular area. Thus, in our work, we use an RNN model [ 15 ], which aims to analyze the slow and long closure of the driver’s eyes. This approach uses a Haar-like descriptor [ 19 ] and an AdaBoost classification algorithm [ 20 ] for face and eye-tracking by using Percent Eye closure (PERCLOS) to evaluate driver tiredness.…”

Perception Sensors for Road Applications

“…Perhaps the least intrusive, physiology-based approach is to remotely monitor driver drowsiness by using cameras to detect facial expressions. A multi-timescale drowsiness characterization system composed of four binary drowsiness classifiers, operating at four distinct timescales and trained jointly, was developed in Reference [9].…”