Perspective Preserving Solution for Quasi-Orthoscopic Video See-Through HMDs

Cutolo, Fabrizio; Fontana, Umberto; Ferrari, Vincenzo

doi:10.3390/technologies6010009

Cited by 27 publications

(16 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Provided that the virtual information is properly registered with the patient, the use of the VST paradigm allows a pixel-wise digital blending between the computer-generated information and the camera-mediated view of the real patient. When properly implemented (i.e., ensuring low motion-to-photon latency and quasi-orthoscopic visualization of the working area [31][32][33]), VST introduces only negligible chromatic, temporal, and perspective alterations with respect to the naked-eye view that generally does not affect the manual performances [32], at least for short-term use. However, the occlusion of the direct view of the real anatomy raises safety concerns in case of system failure.…”

Section: Theorymentioning

confidence: 99%

Architecture of a Hybrid Video/Optical See-through Head-Mounted Display-Based Augmented Reality Surgical Navigation Platform

et al. 2022

Self Cite

View full text Add to dashboard Cite

In the context of image-guided surgery, augmented reality (AR) represents a ground-breaking enticing improvement, mostly when paired with wearability in the case of open surgery. Commercially available AR head-mounted displays (HMDs), designed for general purposes, are increasingly used outside their indications to develop surgical guidance applications with the ambition to demonstrate the potential of AR in surgery. The applications proposed in the literature underline the hunger for AR-guidance in the surgical room together with the limitations that hinder commercial HMDs from being the answer to such a need. The medical domain demands specifically developed devices that address, together with ergonomics, the achievement of surgical accuracy objectives and compliance with medical device regulations. In the framework of an EU Horizon2020 project, a hybrid video and optical see-through augmented reality headset paired with a software architecture, both specifically designed to be seamlessly integrated into the surgical workflow, has been developed. In this paper, the overall architecture of the system is described. The developed AR HMD surgical navigation platform was positively tested on seven patients to aid the surgeon while performing Le Fort 1 osteotomy in cranio-maxillofacial surgery, demonstrating the value of the hybrid approach and the safety and usability of the navigation platform.

show abstract

Section: Theorymentioning

confidence: 99%

Architecture of a Hybrid Video/Optical See-through Head-Mounted Display-Based Augmented Reality Surgical Navigation Platform

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Image stitching can be performed by applying an appropriate warping of the camera images based on the estimation of the three plane-induced homographies between each camera image and a virtual camera, ideally placed at their barycenter [28]. This allows us to remap each camera view on an ideal and central viewpoint of the operatory site plane.…”

Section: Methodsmentioning

confidence: 99%

Low-Computational Cost Stitching Method in a Three-Eyed Endoscope

Mamone

Condino

Cutolo

et al. 2019

Journal of Healthcare Engineering

Self Cite

View full text Add to dashboard Cite

Aortic valve replacement is the only definitive treatment for aortic stenosis, a highly prevalent condition in elderly population. Minimally invasive surgery brought numerous benefits to this intervention, and robotics recently provided additional improvements in terms of telemanipulation, motion scaling, and smaller incisions. Difficulties in obtaining a clear and wide field of vision is a major challenge in minimally invasive aortic valve surgery: surgeon orientates with difficulty because of lack of direct view and limited spaces. This work focuses on the development of a computer vision methodology, for a three-eyed endoscopic vision system, to ease minimally invasive instrument guidance during aortic valve surgery. Specifically, it presents an efficient image stitching method to improve spatial awareness and overcome the orientation problems which arise when cameras are decentralized with respect to the main axis of the aorta and are nonparallel oriented. The proposed approach was tested for the navigation of an innovative robotic system for minimally invasive valve surgery. Based on the specific geometry of the setup and the intrinsic parameters of the three cameras, we estimate the proper plane-induced homographic transformation that merges the views of the operatory site plane into a single stitched image. To evaluate the deviation from the image correct alignment, we performed quantitative tests by stitching a chessboard pattern. The tests showed a minimum error with respect to the image size of 0.46 ± 0.15% measured at the homography distance of 40 mm and a maximum error of 6.09 ± 0.23% at the maximum offset of 10 mm. Three experienced surgeons in aortic valve replacement by mini-sternotomy and mini-thoracotomy performed experimental tests based on the comparison of navigation and orientation capabilities in a silicone aorta with and without stitched image. The tests showed that the stitched image allows for good orientation and navigation within the aorta, and furthermore, it provides more safety while releasing the valve than driving from the three separate views. The average processing time for the stitching of three views into one image is 12.6 ms, proving that the method is not computationally expensive, thus leaving space for further real-time processing.

show abstract

“…The see-through AR headset was designed and assembled by re-engineering and reworking a commercial binocular OST visor (ARS.30 by Trivisio [33]) with a similar approach of our previous works [34,35] (Figure 1). The main characteristics of the headset are listed below.…”

Section: Hardwarementioning

confidence: 99%

Ambiguity-Free Optical–Inertial Tracking for Augmented Reality Headsets

Cutolo

Mamone

Carbonaro

et al. 2020

Sensors

Self Cite

View full text Add to dashboard Cite

The increasing capability of computing power and mobile graphics has made possible the release of self-contained augmented reality (AR) headsets featuring efficient head-anchored tracking solutions. Ego motion estimation based on well-established infrared tracking of markers ensures sufficient accuracy and robustness. Unfortunately, wearable visible-light stereo cameras with short baseline and operating under uncontrolled lighting conditions suffer from tracking failures and ambiguities in pose estimation. To improve the accuracy of optical self-tracking and its resiliency to marker occlusions, degraded camera calibrations, and inconsistent lighting, in this work we propose a sensor fusion approach based on Kalman filtering that integrates optical tracking data with inertial tracking data when computing motion correlation. In order to measure improvements in AR overlay accuracy, experiments are performed with a custom-made AR headset designed for supporting complex manual tasks performed under direct vision. Experimental results show that the proposed solution improves the head-mounted display (HMD) tracking accuracy by one third and improves the robustness by also capturing the orientation of the target scene when some of the markers are occluded and when the optical tracking yields unstable and/or ambiguous results due to the limitations of using head-anchored stereo tracking cameras under uncontrollable lighting conditions.

show abstract

Perspective Preserving Solution for Quasi-Orthoscopic Video See-Through HMDs

Cited by 27 publications

References 34 publications

Architecture of a Hybrid Video/Optical See-through Head-Mounted Display-Based Augmented Reality Surgical Navigation Platform

Architecture of a Hybrid Video/Optical See-through Head-Mounted Display-Based Augmented Reality Surgical Navigation Platform

Low-Computational Cost Stitching Method in a Three-Eyed Endoscope

Ambiguity-Free Optical–Inertial Tracking for Augmented Reality Headsets

Contact Info

Product

Resources

About