A Survey of Calibration Methods for Optical See-Through Head-Mounted Displays

Abstract

Optical see-through head-mounted displays (OST HMDs) are a major output medium for Augmented Reality, which have seen significant growth in popularity and usage among the general public due to the growing release of consumer-oriented models, such as the Microsoft Hololens. Unlike Virtual Reality headsets, OST HMDs inherently support the addition of computer-generated graphics directly into the light path between a user's eyes and their view of the physical world. As with most Augmented and Virtual Reality systems, the physical position of an OST HMD is typically determined by an external or embedded 6-Degree-of-Freedom tracking system. However, in order to properly render virtual objects, which are perceived as spatially aligned with the physical environment, it is also necessary to accurately measure the position of the user's eyes within the tracking system's coordinate frame. For over 20 years, researchers have proposed various calibration methods to determine this needed eye position. However, to date, there has not been a comprehensive overview of these procedures and their requirements. Hence, this paper surveys the field of calibration methods for OST HMDs. Specifically, it provides insights into the fundamentals of calibration techniques, and presents an overview of both manual and automatic approaches, as well as evaluation methods and metrics. Finally, it also identifies opportunities for future research.

A hand‐eye calibration method for augmented reality applied to computer‐assisted orthopedic surgery

“…Over the past few years, the computer science community has made considerable efforts towards the development of calibration techniques for wearable optical see‐through head‐mounted displays (OST‐HMDs), which still remains a challenging and tailored problem due to the complexities associated with specific image projection systems, optics, as well as to neurophysiological features of the human visual system. Currently, available solutions to determine computational relationships between the OST‐HMDs imaging system and the user's eyes are established based on image‐vision methods and external tracking systems, which are cumbersome and error‐prone, largely relying on subjective assessments provided by the feedback from users …”

Off-Line Camera-Based Calibration for Optical See-Through Head-Mounted Displays

“…Research on how to achieve correct calibration of OST displays has been conducted over many years. In 2017, Grubert et al [7] presented a comprehensive survey of all the calibration procedures proposed up to that time. In their work, the authors provided also useful insights into the fundamentals of calibration techniques, grouping them into three main categories: manual methods, semi-automatic methods, and automatic methods.…”

“…Overall, in visual AR applications, the problem of defining the appropriate spatial location of the digital 3D content with respect to the real scene is the principal factor that provides the user with a sense of perceptual congruity [7]. This problem is particularly challenging in OST HMDs, for the solution of which, knowing of the position of the user's viewpoint(s) cannot be overlooked.…”

Perspective Preserving Solution for Quasi-Orthoscopic Video See-Through HMDs

“…encapsulates the parameters of the near-eye display's frustum and it is approximately derived from the specifics of the HMD as follows [28].…”

“…K C encapsulates the camera intrinsic parameters and it is computed by using the Zhang's method [27] implemented within the camera calibrator tool of MATLAB. K D encapsulates the parameters of the near-eye display's frustum and it is approximately derived from the specifics of the HMD as follows [28]. We derived the focal length of the display ( f ) by using the factory specifics of the horizontal and vertical FoV of the display.…”