Corneal-Imaging Calibration for Optical See-Through Head-Mounted Displays

Abstract

In recent years optical see-through head-mounted displays (OST-HMDs) have moved from conceptual research to a market of mass-produced devices with new models and applications being released continuously. It remains challenging to deploy augmented reality (AR) applications that require consistent spatial visualization. Examples include maintenance, training and medical tasks, as the view of the attached scene camera is shifted from the user's view. A calibration step can compute the relationship between the HMD-screen and the user's eye to align the digital content. However, this alignment is only viable as long as the display does not move, an assumption that rarely holds for an extended period of time. As a consequence, continuous recalibration is necessary. Manual calibration methods are tedious and rarely support practical applications. Existing automated methods do not account for user-specific parameters and are error prone. We propose the combination of a pre-calibrated display with a per-frame estimation of the user's cornea position to estimate the individual eye center and continuously recalibrate the system. With this, we also obtain the gaze direction, which allows for instantaneous uncalibrated eye gaze tracking, without the need for additional hardware and complex illumination. Contrary to existing methods, we use simple image processing and do not rely on iris tracking, which is typically noisy and can be ambiguous. Evaluation with simulated and real data shows that our approach achieves a more accurate and stable eye pose estimation, which results in an improved and practical calibration with a largely improved distribution of projection error.

Light-Field Correction for Spatial Calibration of Optical See-Through Head-Mounted Displays

“…Secondly, the visual axis of the human eye differs from the optical axis in the two-sphere eye model we employed [28] This issue requires actual users. A camera integrated in a prosthetic eye might be an alter-native, yet we have no clue how accurately such a system can mimic the real eye.…”

“…Future work directions involve: considering the distortion of the virtual screen [20,21,27] which is assumed to be planer in this paper, deepening the understanding of the eye-dependent parameters [28], investigating the possibility of automated frame-wise OST-HMD calibrations, establishing and refining ways to compare different calibration methods with both subjective [24] and objective error measurements, overcoming the latency issue which is also another dominant aspects directly affects to the spatial registration quality [37], and so on.…”

Eyemirror

“…We used six different modes for gaze estimation on the projected display: EyeMirror-Pupil (EM-P), transforming the pupil center using a homography, and EyeMirror-Cluster (EM-C) taking the cluster of the key feature pairs as gaze point, both described above; EyeMirror-Pupil-Undistorted (EM-P-U) and EyeMirror-Cluster-Undistorted (EM-C-U), both using a corrected corneal image to investigate the effect of the spherical distortion of the corneal reflection (as in [31]); Marker Tracking (MT), using a set of visual markers shown on the screen to track the orientation between the display and the eye tracker provided by the Pupil framework 4 ; and a simple Head Orientation (HO) approach, tracking the participant's head with a Kinect v2 sensor, placed underneath the projected display. For MT we calibrated the eye tracker for each participant separately from the centered location in front of the display.…”

eyeSelfie

“…While advantageous in some applications by adding comfort and usability, a large eye box can sometimes be undesirable Plopski et al 2015]. The large eye box of consumer HMDs allows the user to be aligned imperfectly, causing unwanted distortions in the viewed image and may contribute to user nausea.…”