Vision-Based Instant Measurement System for Driver Fatigue Monitoring

Abstract

In this paper, a vision-based physiological signal measurement system is proposed to instantly measure driver fatigue. A remote photoplethysmography (rPPG) signal is a type of physiological signal measured by a camera without any contact device, and it also retains the characteristics of the PPG, which is useful to evaluate fatigue. To solve the inconvenience caused by the traditional contact-based physiological fatigue detection system and to improve the accuracy, the system measures both the motional and physiological information by using one image sensor. In a practical application, the environmental noise would affect the measured signal, and therefore, we performed a preprocessing step on the signal to extract a clear signal. The experiment was designed in collaboration with Taipei Medical University, and a questionnaire-based method was used to define fatigue. The questionnaire that could directly react to the feeling of the subject was treated as our ground truth. The evaluated correlation was 0.89 and the root mean square error was 0.65 for ten-fold cross-validation on the dataset. The trend of driver fatigue could be evaluated without a contact device by the proposed system. This advantage ensures the safety of the driver and reliability of the system.INDEX TERMS Fatigue monitoring, remote photoplethysmography, biomedical monitoring, image sequence analysis.

Active Vision-Based Attention Monitoring System for Non-Distracted Driving

“…Wang et al [6] was prepared using a eye tracker. Similar to us [7]- [9], [11] also used a normal camera, but camera used by Tsai et al [11] and Shibli et al [9] was expensive than the proposed system. • Although Alam et al [7], [8] used the similar parameters to estimate the attention state but they did not develop any alarm system to alert the driver in the lower level of attention during driving.…”

“…Most of these researches investigated either distraction or sleepiness using only one or two visual parameters for detection of drivers' attention. To solve the inconvenience caused by physiological approaches in [11] a vision-based physiological signal measurement system was proposed to estimate driver fatigue. The system uses only one camera to collect physiological information (e.g.…”

“…Vision-based attentional information includes facial features and body movements. Most used non-invasive attentional cues in currently available vision-based systems are eyelid movements (e.g.,eye blink frequency, closure duration) [6]- [11], eye gaze [6]- [8], [12], head movement [7]- [9], [13], [14], facial expressions (e.g., yawning, lip movements etc.) [7], [11], [13], [15], and body movements (like hand movement) [15]- [17].…”

“…Most used non-invasive attentional cues in currently available vision-based systems are eyelid movements (e.g.,eye blink frequency, closure duration) [6]- [11], eye gaze [6]- [8], [12], head movement [7]- [9], [13], [14], facial expressions (e.g., yawning, lip movements etc.) [7], [11], [13], [15], and body movements (like hand movement) [15]- [17]. However, these existing vision-based systems have some limitations: (i) capturing sensors used by the aforementioned systems are either expensive camera(s) [7]- [10], [14], [15], [17] with any additional sensor/hardware [11], [12], [16], [17] or some specialized imaging sensor (e.g., eye tracker [6] and Kinect [13]); (ii) some of the systems used only single parameters such as pupil [12], PER-CLOS [10] and head pose [14] to estimate driver's attentional state making the system unable to adapt to some situations which are common in the real driving scenario (for example, turning head or wearing sun glass can hide eye) and resulting in incorrect attentional state detection; (iii) some existing systems detect single inattentional state [8]- [14], [17] or limited only to the level of the same state [6], whereas some other focus on detecting driver's activity [15], [16]; (iv) some systems do not have any alert system to warn the driver of any inattentional state when detected during driving [6]- [8], …”

“…[7], [11], [13], [15], and body movements (like hand movement) [15]- [17]. However, these existing vision-based systems have some limitations: (i) capturing sensors used by the aforementioned systems are either expensive camera(s) [7]- [10], [14], [15], [17] with any additional sensor/hardware [11], [12], [16], [17] or some specialized imaging sensor (e.g., eye tracker [6] and Kinect [13]); (ii) some of the systems used only single parameters such as pupil [12], PER-CLOS [10] and head pose [14] to estimate driver's attentional state making the system unable to adapt to some situations which are common in the real driving scenario (for example, turning head or wearing sun glass can hide eye) and resulting in incorrect attentional state detection; (iii) some existing systems detect single inattentional state [8]- [14], [17] or limited only to the level of the same state [6], whereas some other focus on detecting driver's activity [15], [16]; (iv) some systems do not have any alert system to warn the driver of any inattentional state when detected during driving [6]- [8], [10], [11], [14]- [16]; (v) some of the previous works [11], [15], [17] were evaluated in a simulated environment and may not work accurately in the real driving scenario; and (vi) no evidence was provided about the systems mentioned above working in diverse situations (e.g. drivers were having different facial features such as beard, moustache, and hairstyles or wearing accessories (e.g.…”