Raycast Calibration for Augmented Reality HMDs with Off-Axis Reflective Combiners

Guo, Qi; Tang, Huixuan; Schmitz, Aaron J.; Zhang, Wenqi; Lou, Yang; Fix, Alexander; Lovegrove, Steven; Strasdat, Hauke

doi:10.1109/iccp48838.2020.9105134

Cited by 2 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent works [13], [14] rely on differentiable ray tracing for end-to-end designs in computational imaging. Similar trends also appear in other research domains for solving inverse problems utilizing this additional amount of information on derivatives, in computer graphics [37], 3D reconstruction [38], ptychography [39], [40], head-mounted displays calibration [41], lens metrology [42], and phase microscopy [43].…”

Section: Introductionsupporting

confidence: 66%

dO: A Differentiable Engine for Deep Lens Design of Computational Imaging Systems

Wang

Heidrich

2022

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Computational imaging systems algorithmically post-process acquisition images either to reveal physical quantities of interest or to increase image quality, e.g., deblurring. Designing a computational imaging system requires co-design of optics and algorithms, and recently Deep Lens systems have been proposed in which both components are end-to-end designed using data-driven end-to-end training. However, progress on this exciting concept has so far been hampered by the lack of differentiable forward simulations for complex optical design spaces.Here, we introduce dO (DiffOptics) to provide derivative insights into the design pipeline to chain variable parameters and their gradients to an error metric through differential ray tracing. However, straightforward back-propagation of many millions of rays requires unaffordable device memory, and is not resolved by prior works. dO alleviates this issue using two customized memory-efficient techniques: differentiable raysurface intersection and adjoint back-propagation. Broad application examples demonstrate the versatility and flexibility of dO, including classical lens designs in asphere, double-Gauss, and freeform, reverse engineering for metrology, and joint designs of optics-network in computational imaging applications. We believe dO enables a radically new approach to computational imaging system designs and relevant research domains.

show abstract

Section: Introductionsupporting

confidence: 66%

dO: A Differentiable Engine for Deep Lens Design of Computational Imaging Systems

Wang

Heidrich

2022

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

show abstract

“…Achieving a wide field of view (FoV), a key requirement for NEDs to enhance immersion [1], tends to compromise the form factor [2] or the image distortion to employ image expanding optics. Modern NEDs employ a variety of off-axis optics for wide FoV such as curved beam splitter [3][4][5], Holographic Optical Elements (HoE) [6,7], and polarization-based pancake optics [8,9]. While they improve the FoV and potentially reduce the display form factor, these complex, off-axis optics cause non-linear, viewpoint-dependent image distortion (Fig.…”

Section: Introductionmentioning

confidence: 99%

“…On the other hand, ray-tracing-based approaches [5,10] model image distortion by simulating the multi-stage refraction and reflection of the light rays passing through the optical system. Although these methods achieve high accuracy, they take a lot of computation time and are not practical for interactive VR/AR applications.…”

Section: Introductionmentioning

confidence: 99%

Neural distortion fields for spatial calibration of wide field-of-view near-eye displays

Hiroi¹,

Someya

Itoh³

2022

Opt. Express

View full text Add to dashboard Cite

We propose a spatial calibration method for wide field-of-view (FoV) near-eye displays (NEDs) with complex image distortions. Image distortions in NEDs can destroy the reality of the virtual object and cause sickness. To achieve distortion-free images in NEDs, it is necessary to establish a pixel-by-pixel correspondence between the viewpoint and the displayed image. Designing compact and wide-FoV NEDs requires complex optical designs. In such designs, the displayed images are subject to gaze-contingent, non-linear geometric distortions, which explicit geometric models can be difficult to represent or computationally intensive to optimize. To solve these problems, we propose neural distortion field (NDF), a fully-connected deep neural network that implicitly represents display surfaces complexly distorted in spaces. NDF takes spatial position and gaze direction as input and outputs the display pixel coordinate and its intensity as perceived in the input gaze direction. We synthesize the distortion map from a novel viewpoint by querying points on the ray from the viewpoint and computing a weighted sum to project output display coordinates into an image. Experiments showed that NDF calibrates an augmented reality NED with 90° FoV with about 3.23 pixel (5.8 arcmin) median error using only 8 training viewpoints. Additionally, we confirmed that NDF calibrates more accurately than the non-linear polynomial fitting, especially around the center of the FoV.

show abstract

Raycast Calibration for Augmented Reality HMDs with Off-Axis Reflective Combiners

Cited by 2 publications

References 21 publications

dO: A Differentiable Engine for Deep Lens Design of Computational Imaging Systems

dO: A Differentiable Engine for Deep Lens Design of Computational Imaging Systems

Neural distortion fields for spatial calibration of wide field-of-view near-eye displays

Contact Info

Product

Resources

About