Deep learning-driven catheter tracking from bi-plane X-ray fluoroscopy of 3D printed heart phantoms

Abstract

Minimally invasive surgery (MIS) has changed not only the performance of specific operations but also the more effective strategic approach to all surgeries. Expansion of MIS to more complex surgeries demands further development of new technologies, including robotic surgical systems, navigation, guidance, visualizations, dexterity enhancement, and 3D printing technology. In the cardiovascular domain, 3D printed modeling can play a crucial role in providing improved visualization of the anatomical details and guide precision operations as well as functional evaluation of various congenital and congestive heart conditions. In this work, we propose a novel deep learning-driven tracking method for providing quantitative 3D tracking of mock cardiac interventions on customdesigned 3D printed heart phantoms. In this study, the position of the tip of a catheter is tracked from bi-plane fluoroscopic images. The continuous positioning of the catheter relative to the 3D printed model was co-registered in a single coordinate system using external fiducial markers embedded into the model. Our proposed method has the potential to provide quantitative analysis for training exercises of percutaneous procedures guided by bi-plane fluoroscopy.

Topology observing 3D device reconstruction from continuous‐sweep limited angle fluoroscopy

“…The final 3D location p p p i of the control point is then determined by finding the candidate point c c c i (𝜆) with 𝜆 ∈ Λ i that minimizes the distance to the 3D reconstruction of the device from the previous frame, where 𝜙 𝜙 𝜙 (t−1) k is the k th out of n points along the cubic spline describing the previous reconstruction. (12) with…”

“…The device is segmented in both images and the combined information is used to generate a 3D representation. [6][7][8][9][10][11][12][13] This technique has also been extended to other types of devices including prosthetic heart valves. 13 A limitation of this "two view" reconstruction approach is the need for biplane C-arm systems, which are not as widely available as single plane systems.…”

“…4D fluoroscopy can be achieved using biplane C‐arm systems, where two fluoroscopic images are acquired simultaneously from different angles. The device is segmented in both images and the combined information is used to generate a 3D representation 6–13 . This technique has also been extended to other types of devices including prosthetic heart valves 13 .…”

A Coordinate-Regression-Based Deep Learning Model for Catheter Detection during Structural Heart Interventions

“…Our group [15][16][17], among others [18][19][20][21], have shown the promise of augmented and mixed-reality visualization to address the limitations in depth perception for cardiac interventions by using 3D holographic headsets. In order to leverage mixed-reality technology as a navigation tool for fluoroscopy-guided percutaneous procedures, catheter tracking is a cornerstone of the image guidance system.…”

“…To address these limitations with EM sensor tracking, we previously presented a novel deep-learning-driven method for tracking a catheter in a 3D-printed heart phantom from biplane fluoroscopic images that were acquired during a mock procedure [17]. In this work, we trained a U-Net [22] model on the 3D-printed heart phantom to segment a radiopaque marker at the tip of the catheter.…”