Combining Head Pose and Eye Location Information for Gaze Estimation

Abstract

Abstract-Head pose and eye location for gaze estimation have been separately studied in numerous works in the literature. Previous research shows that satisfactory accuracy in head pose and eye location estimation can be achieved in constrained settings. However, in the presence of nonfrontal faces, eye locators are not adequate to accurately locate the center of the eyes. On the other hand, head pose estimation techniques are able to deal with these conditions; hence, they may be suited to enhance the accuracy of eye localization. Therefore, in this paper, a hybrid scheme is proposed to combine head pose and eye location information to obtain enhanced gaze estimation. To this end, the transformation matrix obtained from the head pose is used to normalize the eye regions, and in turn, the transformation matrix generated by the found eye location is used to correct the pose estimation procedure. The scheme is designed to enhance the accuracy of eye location estimations, particularly in low-resolution videos, to extend the operative range of the eye locators, and to improve the accuracy of the head pose tracker. These enhanced estimations are then combined to obtain a novel visual gaze estimation system, which uses both eye location and head information to refine the gaze estimates. From the experimental results, it can be derived that the proposed unified scheme improves the accuracy of eye estimations by 16% to 23%. Furthermore, it considerably extends its operating range by more than 15 by overcoming the problems introduced by extreme head poses. Moreover, the accuracy of the head pose tracker is improved by 12% to 24%. Finally, the experimentation on the proposed combined gaze estimation system shows that it is accurate (with a mean error between 2 and 5 ) and that it can be used in cases where classic approaches would fail without imposing restraints on the position of the head.Index Terms-Eye center location, gaze estimation, head pose estimation. I. MOTIVATION AND RELATED WORK IMAGE-BASED gaze estimation is important in many applications, spanning from human-computer interaction (HCI) to human behavior analysis. In applications where human activity is under observation from a static camera, the There is an abundance of literature concerning these two topics separately; recent surveys on eye center location and head pose estimation can be found in [17] and [31]. The eye location algorithms found in commercially available eye trackers share the problem of sensitivity to head pose variations and require the user to be either equipped with a head-mounted device or to use a high-resolution camera combined with a chin rest to limit the allowed head movement. Furthermore, daylight applications are precluded due to the use of active infrared (IR) illumination to obtain accurate eye location through corneal reflection. The appearance-based methods that make use of standard low-resolution cameras are considered to be less invasive and, thus, more desirable in a large range of applications. Within the appearance-based...

Visualization Analysis of Learning Attention Based on Single-image PnP Head Pose Estimation

“…There are two types: Head-mounted eye tracker [2] and desktop eye tracker [3]. By detecting the eyeball pose and the head pose comprehensively, it can determine the screen area the person sees [4]. Both of them have high accuracy and sensitivity in attention analysis.…”

Continuous Head Movement Estimator for Driver Assistance: Issues, Algorithms, and On-Road Evaluations

“…Methods based on shape features analyze geometric configuration of facial features along with face model (e.g. cylindrical [46], ellipsoidal [47] or mean 3D face [24]) to recover head pose. Smith et al proposed several strategies using global motion and color statistics to detect and track both eyes, lip corners, and the bounding box of the face [48].…”

Robust and continuous estimation of driver gaze zone by dynamic analysis of multiple face videos

“…cylindrical [20], ellipsoidal [21] or mean 3D face [15]) to recover head pose. Smith et al analyzed global motion, and color and intensity statistics to track head and facial features such as eyes, lip corners, and the bounding box of the face [22].…”