Registration-by-regression of coronary CTA and X-ray angiography

Gouveia, Ana Isabel; Metz, Coert; Freire, Luís; Almeida, Pedro; Klein, Stefan

doi:10.1080/21681163.2015.1054520

Cited by 17 publications

(9 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several attempts have been made recently to tackle 2D/3D registration problems using learning-based approaches. Hand crafted features and simple regression modules like Multi-Layer Perceptron and linear regression were proposed to regress 2D/3D registration parameters (Gouveia et al 2015) (Chou et al 2013. A CNN-based regression approach was introduced in (Miao et al 2016) to solve 2D/3D registration for 6 DoF device pose estimation from X-ray images, and later on a domain adaptation scheme was proposed (Zheng et al 2017) to improve the performance generalization of the CNN model on unseen real data.…”

Section: Learning-based Image Registrationmentioning

confidence: 99%

Dilated FCN for Multi-Agent 2D/3D Medical Image Registration

Miao

Piat

Fischer

et al. 2017

Preprint

View full text Add to dashboard Cite

2D/3D image registration to align a 3D volume and 2D X-ray images is a challenging problem due to its ill-posed nature and various artifacts presented in 2D X-ray images. In this paper, we propose a multi-agent system with an auto attention mechanism for robust and efficient 2D/3D image registration. Specifically, an individual agent is trained with dilated Fully Convolutional Network (FCN) to perform registration in a Markov Decision Process (MDP) by observing a local region, and the final action is then taken based on the proposals from multiple agents and weighted by their corresponding confidence levels. The contributions of this paper are threefold. First, we formulate 2D/3D registration as a MDP with observations, actions, and rewards properly defined with respect to X-ray imaging systems. Second, to handle various artifacts in 2D X-ray images, multiple local agents are employed efficiently via FCN-based structures, and an auto attention mechanism is proposed to favor the proposals from regions with more reliable visual cues. Third, a dilated FCN-based training mechanism is proposed to significantly reduce the Degree of Freedom in the simulation of registration environment, and drastically improve training efficiency by an order of magnitude compared to standard CNN-based training method. We demonstrate that the proposed method achieves high robustness on both spine cone beam Computed Tomography data with a low signal-to-noise ratio and data from minimally invasive spine surgery where severe image artifacts and occlusions are presented due to metal screws and guide wires, outperforming other state-of-the-art methods (single agent-based and optimization-based) by a large margin. Computed Tomography (CT), Mangnetic Resonance Imaging (MRI) etc), to align its projections with given 2D Xray images. Reliable 2D/3D registration is a key enabler for image-guided surgeries in modern operating rooms. It brings measurement and plannings done on the pre-operative data into the operating room, and fuse it with intra-operative live 2D X-ray images. It can be used to provide augmented reality image guidance for the surgery, or provide navigation for robotic surgery. Despite that 2D/3D registration has been

show abstract

Section: Learning-based Image Registrationmentioning

confidence: 99%

Dilated FCN for Multi-Agent 2D/3D Medical Image Registration

Miao

Piat

Fischer

et al. 2017

Preprint

View full text Add to dashboard Cite

show abstract

“…Due to the requirement of the segmentation, these methods are only suitable for applications where the target object has high contrast in fluoroscopic images so that it can be reliably segmented (e.g., metal implants). Instead of directly matching input image with templates, in [14] a multilayer perceptron (MLP) regressor is trained from the templates to approximate the mapping from image to pose parameters. Since the complex template matching process is approximated by a MLP regressor, the registration accuracy reported in [14] is lower than that achievable using other template matching methods and 2D/3D registration methods.…”

Section: Introductionmentioning

confidence: 99%

“…Instead of directly matching input image with templates, in [14] a multilayer perceptron (MLP) regressor is trained from the templates to approximate the mapping from image to pose parameters. Since the complex template matching process is approximated by a MLP regressor, the registration accuracy reported in [14] is lower than that achievable using other template matching methods and 2D/3D registration methods. Therefore, the MLP regression method is more suitable to be used as a coarse initialization step to start 2D/3D registration, which is typically more accurate but with a small capture range.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Real-time 6DoF pose recovery from X-ray images using library-based DRR and hybrid optimization

et al. 2016

View full text Add to dashboard Cite

show abstract

“…To avoid computationally expensive optimization, the learning‐based methods constructed a nonlinear mapping function from an image pair to the volumetric deformation field. The mapping functions can be modeled by the multilayer perception 20 and the multiscale linear regression model 21 . The partial least squares regression (PLSR) 22 and the regression random forest 23 have been used in the 2D–3D registration.…”

Section: Introductionmentioning

confidence: 99%

Deformable registration of lateral cephalogram and cone‐beam computed tomography image

Zhang

Qin

et al. 2021

Medical Physics

View full text Add to dashboard Cite

Purpose: This study aimed to design and evaluate a novel method for the registration of 2D lateral cephalograms and 3D craniofacial cone‐beam computed tomography (CBCT) images, providing patient‐specific 3D structures from a 2D lateral cephalogram without additional radiation exposure. Methods: We developed a cross‐modal deformable registration model based on a deep convolutional neural network. Our approach took advantage of a low‐dimensional deformation field encoding and an iterative feedback scheme to infer coarse‐to‐fine volumetric deformations. In particular, we constructed a statistical subspace of deformation fields and parameterized the nonlinear mapping function from an image pair, consisting of the target 2D lateral cephalogram and the reference volumetric CBCT, to a latent encoding of the deformation field. Instead of the one‐shot registration by the learned mapping function, a feedback scheme was introduced to progressively update the reference volumetric image and to infer coarse‐to‐fine deformations fields, accounting for the shape variations of anatomical structures. A total of 220 clinically obtained CBCTs were used to train and validate the proposed model, among which 120 CBCTs were used to generate a training dataset with 24k paired synthetic lateral cephalograms and CBCTs. The proposed approach was evaluated on the deformable 2D–3D registration of clinically obtained lateral cephalograms and CBCTs from growing and adult orthodontic patients. Results: Strong structural consistencies were observed between the deformed CBCT and the target lateral cephalogram in all criteria. The proposed method achieved state‐of‐the‐art performances with the mean contour deviation of 0.41±0.12 mm on the anterior cranial base, 0.48±0.17 mm on the mandible, and 0.35±0.08 mm on the maxilla, respectively. The mean surface mesh ranged from 0.78 to 0.97 mm on various craniofacial structures, and the LREs ranged from 0.83 to 1.24 mm on the growing datasets regarding 14 landmarks. The proposed iterative feedback scheme handled the structural details and improved the registration. The resultant deformed volumetric image was consistent with the target lateral cephalogram in both 2D projective planes and 3D volumetric space regarding the multicategory craniofacial structures. Conclusions: The results suggest that the deep learning‐based 2D–3D registration model enables the deformable alignment of 2D lateral cephalograms and CBCTs and estimates patient‐specific 3D craniofacial structures.

show abstract

Registration-by-regression of coronary CTA and X-ray angiography

Cited by 17 publications

References 34 publications

Dilated FCN for Multi-Agent 2D/3D Medical Image Registration

Dilated FCN for Multi-Agent 2D/3D Medical Image Registration

Real-time 6DoF pose recovery from X-ray images using library-based DRR and hybrid optimization

Deformable registration of lateral cephalogram and cone‐beam computed tomography image

Contact Info

Product

Resources

About