Catheter pose-dependent virtual angioscopy images visualized on augmented reality glasses

Abstract

Fluoroscopy and digital subtraction angiography provide guidance in endovascular aortic repair (EVAR) but introduce radiation exposure and require the administration of contrast agent. To overcome these disadvantages, previous studies proposed to display the pose of an electromagnetically (EM) tracked catheter tip within a three-dimensional virtual aorta on augmented reality (AR) glasses. For further guidance, we propose to create virtual angioscopy images based on the catheter tip pose within the aorta and to display them on HoloLens. The aorta was segmented from the computed tomography (CT) data using MeVisLab software. A landmarkbased registration allowed the calculation of the pose of the EM sensor in the CT coordinate system. The sensor pose was sent to MeVisLab running on a computer and a virtual angioscopy image was created at runtime based on the segmented aorta. When requested by HoloLens, the last encoded image was sent from MeVisLab to the AR glasses via Wi-Fi using a remote procedure call (gRPC), and then decoded and displayed on HoloLens. For evaluation purposes, the latency of transmitting and displaying the images was measured using two different lossy compression formats (namely JPEG and DXT1). A mean latency of 82 ms was measured for the JPEG format. On the other hand, using the DXT1 format, the mean latency was reduced by 87 %. This study proved the feasibility of creating pose-dependent virtual angioscopy images and displaying them on HoloLens. Additionally, the results showed that the DXT1 format outperformed the JPEG format regarding latency. The virtual angioscopy may add valuable additional information for guidance in radiation-sparing EVAR procedure approaches.

Catheter pose-dependent virtual angioscopy images for endovascular aortic repair: validation with a video graphics array (VGA) camera

“…The EM data is sent from the EM tracking system to the NavEvar application via the open-source software PLUS Toolkit (https://plustoolkit.github. io) [2]. The virtual camera produces images of 512 × 512 pixels and the field of view (in degrees) can be adjusted.…”

“…To reduce EVAR limitations, a recent study reported the integration of a multicore fiber with fiber Bragg gratings and two electromagnetic (EM) sensors into a stent graft system to locate the shape of the endovascular tool with respect to the patient's anatomy (in that case, a patient-specific aortic phantom) [1]. In addition, virtual angioscopy images were rendered based on the pose (position and orientation) of an EM-tracked catheter tip [2]. These images of the vessel lumen were visualised on a virtual 2D canvas superimposed on the real scenario with the augmented reality glasses Microsoft HoloLens (first generation).…”

“…These images of the vessel lumen were visualised on a virtual 2D canvas superimposed on the real scenario with the augmented reality glasses Microsoft HoloLens (first generation). These previous studies [1,2] are part of our ongoing research project Nav EVAR.…”

“…These studies showed virtual images and also video images acquired with an endoscope. However, in [2] no validation with video images was conducted so the point of view of the virtual camera was not similar to that of a video camera.…”

“…The objective of this study was to continue our work from [2] comparing virtual angioscopy images with video images using an EVAR patient-specific phantom. The spatial relationship between the EM sensor (coordinate system of the virtual camera) and a video graphics array (VGA) camera was computed with a Hand-Eye calibration so that the virtual camera and the VGA camera produced similar images.…”