Deep Learning Assisted Automatic Intra-operative 3D Aortic Deformation Reconstruction

Zhang, Yanhao; Falque, Raphael; Zhao, Liang; Huang, Shoudong; Hu, Boni

doi:10.1007/978-3-030-59719-1_64

Cited by 5 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…algorithms which require a good prior in order to solve the alignment of a template and a target as a gradient descent optimization. This surprisingly specific problem formulation finds applications in medical robotics for organs alignment [5], clothing manipulation [6], humans [7] and animals [8] body shape reconstruction. Furthermore, given a set of keypoints matching pairs, the rigid alignment of two shapes can be solved in a closed-form with singular value decomposition (SVD) [9] as an alternative to iterative closest point (ICP) methods [10].…”

Section: Introductionmentioning

confidence: 99%

Semantic keypoint extraction for scanned animals using multi-depth-camera systems

Falque¹,

Vidal-Calleja²,

Alempijevic³

2022

Preprint

View full text Add to dashboard Cite

Keypoint annotation in point clouds is an important task for 3D reconstruction, object tracking and alignment, in particular in deformable or moving scenes. In the context of agriculture robotics, it is a critical task for livestock automation to work toward condition assessment or behaviour recognition. In this work, we propose a novel approach for semantic keypoint annotation in point clouds, by reformulating the keypoint extraction as a regression problem of the distance between the keypoints and the rest of the point cloud. We use the distance on the point cloud manifold mapped into a radial basis function (RBF), which is then learned using an encoder-decoder architecture. Special consideration is given to the data augmentation specific to multi-depth-camera systems by considering noise over the extrinsic calibration and camera frame dropout. Additionally, we investigate computationally efficient non-rigid deformation methods that can be applied to animal point clouds. Our method is tested on data collected in the field, on moving beef cattle, with a calibrated system of multiple hardware-synchronised RGB-D cameras.

show abstract

Section: Introductionmentioning

confidence: 99%

Semantic keypoint extraction for scanned animals using multi-depth-camera systems

Falque¹,

Vidal-Calleja²,

Alempijevic³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The state-of-the-art image registration is classified into optimization-based and learning-based approaches. For optimization-based approaches, the aortic centerline [6] and aortic shape contour [7] are commonly used as features for matching. In [6], a graph matching method is proposed to establish the correspondence between the 3D pre-operative and 2D intra-operative skeletons extracting from fluoroscopic images, and then the two skeletons are registered by skeleton deformation.…”

Section: Introductionmentioning

confidence: 99%

“…In [6], a graph matching method is proposed to establish the correspondence between the 3D pre-operative and 2D intra-operative skeletons extracting from fluoroscopic images, and then the two skeletons are registered by skeleton deformation. The work in [7] estimated a warp field of 3D aortic shape deformation by solving a non-linear leastsquares problem based on an embedded deformation graph. However, the optimization-based approaches suffer severely from high computational complexity, and learning-based approaches are explored and reviewed in [8].…”

Section: Introductionmentioning

confidence: 99%

Model-to-Image Registration via Deep Learning towards Image-Guided Endovascular Interventions

Mancini

Monizzi

et al. 2021

2021 International Symposium on Medical Robotics (ISMR)

View full text Add to dashboard Cite

Cardiologists highlight the need for an intraoperative 3D visualization to assist interventions. The intraoperative 2D X-ray/Digital Subtraction Angiography (DSA) images in the standard clinical workflow limit cardiologists' views significantly. Compared with image-to-image registration, model-to-image registration is an essential approach taking advantage of the reuse of pre-operative 3D models reconstructed from Computed Tomography Angiography (CTA) images. Traditional optimized-based registration methods suffer severely from high computational complexity. Moreover, the consequence of lacking ground truth for learning-based registration approaches should not be neglected. To overcome these challenges, we introduce a model-to-image registration framework via deep learning for image-guided endovascular catheterization. This work performs autonomous vessel segmentation from intra-operative fluoroscopy images via a deep residual U-net and a model-to-image matching via a convolutional neural network. For this study, image data were collected from 10 patients who performed Transcatheter Aortic Valve Implantation (TAVI) procedures. It was found that vessel segmentation of test data results in median values of Dice Similarity Coefficient, Precision, and Recall of (0.75, 0.58, 0.67) for femoral artery, and (0.71, 0.56, 0.74) for aortic root. The segmentation network behaves better than manual annotation, and it recognizes part of vessels that were not labeled manually. Image matching between the transformed moving image and the fixed image results in a median value of Recall of 0.90. The proposed approach achieves a good accuracy of vessel segmentation and a good recall value of model-toimage matching.

show abstract

Semantic Keypoint Extraction for Scanned Animals using Multi-Depth-Camera Systems

Falque¹,

Vidal-Calleja²,

Alempijevic³

2023

2023 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Deep Learning Assisted Automatic Intra-operative 3D Aortic Deformation Reconstruction

Cited by 5 publications

References 12 publications

Semantic keypoint extraction for scanned animals using multi-depth-camera systems

Semantic keypoint extraction for scanned animals using multi-depth-camera systems

Model-to-Image Registration via Deep Learning towards Image-Guided Endovascular Interventions

Semantic Keypoint Extraction for Scanned Animals using Multi-Depth-Camera Systems

Contact Info

Product

Resources

About