GraphRegNet: Deep Graph Regularisation Networks on Sparse Keypoints for Dense Registration of 3D Lung CTs

Hansen, Lasse; Heinrich, Mattias P.

doi:10.1109/tmi.2021.3073986

Cited by 50 publications

(17 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our network can be trained within 17 minutes on a single RTX A4000 requiring less than 2 GByte of VRAM, indicating the improved training efficiency with fewer scans when using heatmaps. GraphRegNet [7] is similar in that it also employs heatmaps (integral regression) but more explicitly by defining the exact same discretised displacement grid beforehand and computing an SSD cost tensor based on hand-crafted features as input. While it outperforms our method with a TRE of 1.34mm it appears to be more tailored towards the specific task and might not be easily extendable to end-to-end feature learning or abdominal registration.…”

Section: Discussionmentioning

confidence: 99%

“…This compares very favourable to VoxelMorph+ with 7.98mm and LapIRN with 4.76mm. Of all published DL-methods only GraphRegNet [7] is superior with 1.34 mm. The high visual quality of our registration is shown in Fig.…”

Section: Keypoint Self-supervisionmentioning

confidence: 99%

“…Related work: Addressing large deformations with learning based registration is generally approached by either multi-scale, label-supervised networks [13,20,19] or by employing explicitly discretised displacements [7,9]. Many variants of U-Net like architectures have been proposed that include among others, dual-stream [16], cascades [28] and embeddings [3].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Voxelmorph++ Going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation

Heinrich¹,

Hansen²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The majority of current research in deep learning based image registration addresses inter-patient brain registration with moderate deformation magnitudes. The recent Learn2Reg medical registration benchmark has demonstrated that single-scale U-Net architectures, such as VoxelMorph that directly employ a spatial transformer loss, often do not generalise well beyond the cranial vault and fall short of stateof-the-art performance for abdominal or intra-patient lung registration.Here, we propose two straightforward steps that greatly reduce this gap in accuracy. First, we employ keypoint self-supervision with a novel network head that predicts a discretised heatmap and robustly reduces large deformations for better robustness. Second, we replace multiple learned fine-tuning steps by a single instance optimisation with hand-crafted features and the Adam optimiser. Different to other related work, including FlowNet or PDD-Net, our approach does not require a fully discretised architecture with correlation layer. Our ablation study demonstrates the importance of keypoints in both self-supervised and unsupervised (using only a MIND metric) settings. On a multi-centric inspiration-exhale lung CT dataset, including very challenging COPD scans, our method outperforms VoxelMorph by improving nonlinear alignment by 77% compared to 19% -reaching target registration errors of 2 mm that outperform all but one learning methods published to date. Extending the method to semantic features sets new stat-of-the-art performance on inter-subject abdominal CT registration.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Keypoint Self-supervisionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Voxelmorph++ Going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation

Heinrich¹,

Hansen²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…abdominal registration (L2R task 3). The learning-based counter-parts of probabilistic displacement approaches [9,7] achieved great performances on US-MR brain registration, lung registration, and intra-patient abdominal image fusion but did not reach the same level of performance on inter-patient abdominal registration. Yet, these approaches allow for large deformations by design as they evaluate displacement probabilities within a (large) specified search region.…”

mentioning

confidence: 95%

Driving Points Prediction For Abdominal Probabilistic Registration

Joutard¹,

Dorent²,

Ourselin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Inter-patient abdominal registration has various applications, from pharmakinematic studies to anatomy modeling. Yet, it remains a challenging application due to the morphological heterogeneity and variability of the human abdomen. Among the various registration methods proposed for this task, probabilistic displacement registration models estimate displacement distribution for a subset of points by comparing feature vectors of points from the two images. These probabilistic models are informative and robust while allowing large displacements by design. As the displacement distributions are typically estimated on a subset of points (which we refer to as driving points), due to computational requirements, we propose in this work to learn a driving points predictor. Compared to previously proposed methods, the driving points predictor is optimized in an end-to-end fashion to infer driving points tailored for a specific registration pipeline. We evaluate the impact of our contribution on two different datasets corresponding to different modalities. Specifically, we compared the performances of 6 different probabilistic displacement registration models when using a driving points predictor or one of 2 other standard driving points selection methods. The proposed method improved performances in 11 out of 12 experiments.

show abstract

“…Note that keypoint approaches[16] and approaches based on optimal transport[30] can overcome some of these issues. However, in this work we focus on the registration of images with grid-based displacement fields.…”

mentioning

confidence: 99%

GradICON: Approximate Diffeomorphisms via Gradient Inverse Consistency

Lin¹,

Greer²,

Vialard³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many registration approaches exist with early work focusing on optimizationbased approaches for image pairs. Recent work focuses on deep registration networks to predict spatial transformations. In both cases, commonly used nonparametric registration models, which estimate transformation functions instead of low-dimensional transformation parameters, require choosing a suitable regularizer (to encourage smooth transformations) and its parameters. This makes models difficult to tune and restricts deformations to the deformation space permissible by the chosen regularizer. While deep-learning models for optical flow exist that do not regularize transformations and instead entirely rely on the data these might not yield diffeomorphic transformations which are desirable for medical image registration. In this work, we therefore develop GradICON building upon the unsupervised ICON deep-learning registration approach, which only uses inverse-consistency for regularization. However, in contrast to ICON, we prove and empirically verify that using a gradient inverse-consistency loss not only significantly improves convergence, but also results in a similar implicit regularization of the resulting transformation map. Synthetic experiments and experiments on magnetic resonance (MR) knee images and computed tomography (CT) lung images show the excellent performance of GradICON. We achieve state-of-the-art (SOTA) accuracy while retaining a simple registration formulation, which is practically important.

show abstract

GraphRegNet: Deep Graph Regularisation Networks on Sparse Keypoints for Dense Registration of 3D Lung CTs

Cited by 50 publications

References 45 publications

Voxelmorph++ Going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation

Voxelmorph++ Going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation

Driving Points Prediction For Abdominal Probabilistic Registration

GradICON: Approximate Diffeomorphisms via Gradient Inverse Consistency

Contact Info

Product

Resources

About