Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance

Abstract

T his paper describes a real-time prototype computer vision system for monitoring driver vigilance. The main components of the system consists of a remotely located video CCD camera, a specially designed hardware system for real-time image acquisition and for controlling the illuminator and the alarm system, and various computer vision algorithms for simultaneously, real-time and non-intrusively monitoring various visual bio-behaviors that typically characterize a driver's level of vigilance. The visual behaviors include eyelid movement, face orientation, and gaze movement (pupil movement). The system was tested in a simulating environment with subjects of different ethnic backgrounds, different genders, ages, with/without glasses, and under different illumination conditions, and it was found very robust, reliable and accurate. IntroductionThe ever-increasing number of traffic accidents in the US due to a diminished driver's vigilance level has become a problem of serious concern to society. Drivers with a diminished vigilance level suffer from a marked decline in their abilities of perception, recognition, and vehicle control, and therefore pose serious danger to their own life and the lives of other people. Statistics show that a leading cause for fatal or injury-causing traffic accidents is due to drivers with a diminished vigilance level. In the trucking industry, 57% fatal truck accidents are due to driver fatigue. It is the number 1 cause for heavy truck crashes. Seventy percent of American drivers report driving fatigued. With the ever-growing traffic conditions, this problem will further deteriorate. For this reason, developing systems actively monitoring a driver's level of vigilance and alerting the driver of any insecure driving conditions is essential to prevent accidents.Many efforts [1][2][3][4][5][6][7][8][9] have been reported in the literature for developing active safety systems intended for reducing the number of automobile accidents due to Real-Time Imaging 8, 357-377 (2002Imaging 8, 357-377 ( ) doi:10.1006Imaging 8, 357-377 ( /rtim.2002, available online at http://www.idealibrary.com on reduced vigilance. Among different techniques, the best detection accuracy is achieved with techniques that measure physiological conditions like brain waves, heart rate, and pulse rate [8,10] People in fatigue exhibit certain visual behaviors easily observable from changes in their facial features like the eyes, head, and face. Typical visual characteristics observable from the image of a person with reduced alertness level include slow eyelid movement [12,13], smaller degree of eye openness (or even closed), frequent nodding [14], yawning, gaze (narrowness in the line of sight), sluggish in facial expression, and sagging posture. To make use of these visual cues, another increasingly popular and non-invasive approach for monitoring fatigue is to assess a driver's vigilance level through visual observation of his/her physical conditions using a camera and state-of-the-art technologies in computer vision. ...

On the Roles of Eye Gaze and Head Dynamics in Predicting Driver's Intent to Change Lanes

“…Other studies more recently have included eye gaze measurements as a part of laboratory tests of driver fatigue [19], [20] or of simulated lane change events [21], [22]. Simulators though do not capture all the dynamics and variability of real-world environments [4].…”

A multi-modal architecture for non-intrusive analysis of performance in the workplace

“…This decision was taken after an analysis of existing work that showed the suitability of this kind of algorithms for modeling mental fatigue. Namely, existing works use electroencephalography signals [17], electromyography [18] and other features such as gaze detection or face pose [19] as inputs to neural networks. Existing related work is especially targeted at vehicle driving and operation of machines, which is a classical eld of application of fatigue detection and performance monitoring [19].…”

“…Namely, existing works use electroencephalography signals [17], electromyography [18] and other features such as gaze detection or face pose [19] as inputs to neural networks. Existing related work is especially targeted at vehicle driving and operation of machines, which is a classical eld of application of fatigue detection and performance monitoring [19]. To the extent of our knowledge, it is the rst time that an ANN is used with performance features such as those put forward in this work.…”

Robust and continuous estimation of driver gaze zone by dynamic analysis of multiple face videos

“…To determine head orientation, Ji and Yang [19] used eigenspace algorithm to map pupil related feature to head pose space and further quantized the orientation into seven angles between −45 0 to +45 0 . The pupil itself is obtained from bright and black pupil image acquired using specialized hardware setup.…”