Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks

Abstract

Objective: The focal length of available optical seethrough (OST) head-mounted displays (HMDs) is at least 2 m therefore, during manual tasks, the user eye cannot keep in focus both the virtual and real content at the same time. Another perceptual limitation is related to the vergence-accommodation conflict (VAC), this latter being present in binocular vision only. This paper investigates the effect of incorrect focus cues on the user performance, visual comfort and workload during the execution of augmented reality (AR) guided manual task with one of the most advanced OST HMD, the Microsoft HoloLens. Methods: An experimental study was designed to investigate the performance of 20 subjects in a connect-the-dots task, with and without the use of AR. The following tests were planned: AR guided monocular and binocular; Naked-eye monocular and binocular. Each trial was analyzed to evaluate the accuracy in connecting dots; NASA Task Load Index and Likert questionnaires were used to assess the workload and the visual comfort. Results: No statistically significant differences were found in the workload, and in the perceived comfort between the AR guided binocular and monocular test. User performances were significantly better during the Naked eye tests. No statistically significant differences in performances were found in the monocular and binocular tests. The maximum error in AR tests was 5.9 mm. Conclusion: Even if there is a growing interest in using commercial OST-HMD, for guiding high-precision manual tasks, attention should be paid to the limitations of the available technology not designed for the peripersonal space.

Value of the surgeon’s sightline on hologram registration and targeting in mixed reality

“…Our results fall in this range. On the other hand, our results from the FF scenario on the Y -axis (TED values close to the 6 mm range) corroborate with results obtained by Condino et al [8]. These results emphasize the importance for sightline awareness and correction when using the HoloLens glasses at surgery (Fig.…”

“…Overall, the literature presents error ranges from 0 to 10 mm by using diverse navigation and accuracy measurement methods on different surgical specialties with the Microsoft HoloLens [8,28,29,[32][33][34]. This collective experimental heterogeneity makes it cumbersome to pinpoint the crucial elements that create such data variability.…”

“…At the same time, some accuracy studies on the HoloLens have been disappointing when performing high-precision manual tasks, thus highlighting the limitations of this device and deeming it insufficient [8,9]. From the clinical perspective, surgeons respect the intention-to-treat principle which entails removal of all tumor spreading pathways at surgery [10].…”

Are augmented reality headsets in surgery a dead end?

“…The human eye is not able to accommodate to proper focus both the real world and the virtual content, and this focus rivalry is incompatible with its use as aid to manual activities [9].…”

“…The availability of powerful and user-friendly AR HMDs leveraged the development of many demonstrators for open surgery but without any actual clinical result. In a recent paper, we showed that general-purpose AR HMDs are not yet ready to be routinely used in surgery [9] and the media raised doubtful questions about the dead end of AR HMDs for surgery.…”