Significant acceleration of 2D-3D registration-based fusion of ultrasound and x-ray images by mesh-based DRR rendering

Abstract

For transcatheter-based minimally invasive procedures in structural heart disease ultrasound and X-ray are the two enabling imaging modalities. A live fusion of both real-time modalities can potentially improve the workflow and the catheter navigation by combining the excellent instrument imaging of X-ray with the high-quality soft tissue imaging of ultrasound. A recently published approach to fuse X-ray fluoroscopy with trans-esophageal echo (TEE) registers the ultrasound probe to X-ray images by a 2D-3D registration method which inherently provides a registration of ultrasound images to X-ray images. In this paper, we significantly accelerate the 2D-3D registration method in this context. The main novelty is to generate the projection images (DRR) of the 3D object not via volume ray-casting but instead via a fast rendering of triangular meshes. This is possible, because in the setting for TEE/X-ray fusion the 3D geometry of the ultrasound probe is known in advance and their main components can be described by triangular meshes. We show that the new approach can achieve a speedup factor up to 65 and does not affect the registration accuracy when used in conjunction with the gradient correlation similarity measure. The improvement is independent of the underlying registration optimizer. Based on the results, a TEE/X-ray fusion could be performed with a higher frame rate and a shorter time lag towards real-time registration performance. The approach could potentially accelerate other applications of 2D-3D registrations, e.g. the registration of implant models with X-ray images.Keywords: Mesh-based 2D-3DRegistration, Ultrasound/X-ray Image Fusion, Image Guided Surgery DESCRIPTION OF PURPOSE AND RELATED WORKMore and more procedures in the field of structural heart disease become minimally invasive and catheter-based. This includes for instance trans-catheter aortic valve implantation [1], trans-catheter mitral valve repair [2], closure of atrial septal defects [3], paravalvular leak closure [4] and left atrial appendage occlusion [5]. The drivers for this trend from open-heart surgery to trans-catheter procedures are the availability of new catheter devices and the intra-procedural imaging.Usually these procedures are performed under fluoroscopic X-ray and trans-esophageal echo (TEE). Intra-operatively these modalities are mainly used independently of each other. X-ray imaging is performed by the cardiologist or surgeon at the left or right side of the patient whereas ultrasound imaging is performed by the anesthesiologist at the head side of the patient. An image fusion of both systems could yield a better mutual understanding of the image contents and potentially even allow new kinds of procedures. The images move relatively to each other because the position of the imaging devices is changed by the operator, as well as because of patient, heart and breathing motion. Therefore, there is the demand of an almost real-time update to synchronize the relative position of both images.Several approaches were published fo...

Real-time 6DoF pose recovery from X-ray images using library-based DRR and hybrid optimization

“…Combinations of binary masks have been reported to be used as an alternative of DRR for 2D/3D registration [8]. While high computational efficiency can be achieved, the appearance of binary masks and their combinations is very different from rcDRRs, leading to different similarity profiles that could negatively affect the registration accuracy [9].…”

Robust 5DOF Transesophageal Echo Probe Tracking at Fluoroscopic Frame Rates

“…Aside from the need for custom modification, this solution was undesirable due to its potential to increase the risk of esophageal injury. In [5], DRR generation was accelerated by modeling the TEE probe as a mesh and using OpenGL based rendering. DRR computation was significantly accelerated, but the overall similarity metric computation time (1 ms per iteration) is still not fast enough to perform registration at imaging frame rates based on results reported in this paper.…”