An Unsupervised Learning Model for Deformable Medical Image Registration

Balakrishnan, Guha; Zhao, Amy; Sabuncu, Mert R.; Dalca, Adrian V.; Guttag, John V.

doi:10.1109/cvpr.2018.00964

Cited by 647 publications

(738 citation statements)

References 46 publications

Supporting

Mentioning

710

Contrasting

Order By: Relevance

“…Unsupervised, weakly supervised, and strongly supervised neural networks have been used to estimate deformation vector fields directly from two images. Unsupervised methods learn the deformation directly from pairs of images without a ground truth deformation vector field by maximizing a similarity metric . Strongly supervised methods use a ground truth deformation vector field, usually by applying known transformations to a set of images during training .…”

Section: Introductionmentioning

confidence: 99%

Fast contour propagation for MR‐guided prostate radiotherapy using convolutional neural networks

et al. 2020

View full text Add to dashboard Cite

Purpose To quickly and automatically propagate organ contours from pretreatment to fraction images in magnetic resonance (MR)‐guided prostate external‐beam radiotherapy. Methods Five prostate cancer patients underwent 20 fractions of image‐guided external‐beam radiotherapy on a 1.5 T MR‐Linac system. For each patient, a pretreatment T2‐weighted three‐dimensional (3D) MR imaging (MRI) scan was used to delineate the clinical target volume (CTV) contours. The same scan was repeated during each fraction, with the CTV contour being manually adapted if necessary. A convolutional neural network (CNN) was trained for combined image registration and contour propagation. The network estimated the propagated contour and a deformation field between the two input images. The training set consisted of a synthetically generated ground truth of randomly deformed images and prostate segmentations. We performed a leave‐one‐out cross‐validation on the five patients and propagated the prostate segmentations from the pretreatment to the fraction scans. Three variants of the CNN, aimed at investigating supervision based on optimizing segmentation overlap, optimizing the registration, and a combination of the two were compared to results of the open‐source deformable registration software package Elastix. Results The neural networks trained on segmentation overlap or the combined objective achieved significantly better Hausdorff distances between predicted and ground truth contours than Elastix, at the much faster registration speed of 0.5 s. The CNN variant trained to optimize both the prostate overlap and deformation field, and the variant trained to only maximize the prostate overlap, produced the best propagation results. Conclusions A CNN trained on maximizing prostate overlap and minimizing registration errors provides a fast and accurate method for deformable contour propagation for prostate MR‐guided radiotherapy.

show abstract

Section: Introductionmentioning

confidence: 99%

Fast contour propagation for MR‐guided prostate radiotherapy using convolutional neural networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Two state-of-the-art registration methods, i.e., diffeomorphic demons (D. Demons) [10] and SyN [11], are used as the comparison methods. We also compare our method with other deep learning registration strategies, including 1) supervised training (i.e., ground-truth deformations obtained by SyN), 2) unsupervised training with similarity metrics SSD [4] and CC [5]. …”

Section: Experiments and Resultsmentioning

confidence: 99%

“…In unsupervised learning methods [45], the deformable transformations are learned without ground-truth deformations by maximizing the similarity between a pair of images, such as the sum of squared difference (SSD) and cross-correlation (CC). However, these similarity metrics are closely related to the nature of the images and might not be suitable when dealing with diverse datasets.…”

Section: Introductionmentioning

confidence: 99%

Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning Based Registration

Fan

Cao

Xue

et al. 2018

Lecture Notes in Computer Science

103

View full text Add to dashboard Cite

This paper introduces an unsupervised adversarial similarity network for image registration. Unlike existing deep learning registration frameworks, our approach does not require ground-truth deformations and specific similarity metrics. We connect a registration network and a discrimination network with a deformable transformation layer. The registration network is trained with feedback from the discrimination network, which is designed to judge whether a pair of registered images are sufficiently similar. Using adversarial training, the registration network is trained to predict deformations that are accurate enough to fool the discrimination network. Experiments on four brain MRI datasets indicate that our method yields registration performance that is promising in both accuracy and efficiency compared with state-of-the-art registration methods, including those based on deep learning.

show abstract

“…In synergy with using a CT synth bridge, improvements in mono‐modality registration would also lead to better multimodal registration in the head‐and‐neck. Currently, research using neural networks offers some exciting new avenues in this regard, including completely learning‐based unsupervised DVF generation . However, the performance of these methods depends on the availability and quality of training sets, which are particularly challenging for multimodel registration.…”

Section: Discussionmentioning

confidence: 99%

Multimodality image registration in the head‐and‐neck using a deep learning‐derived synthetic CT as a bridge

et al. 2020

View full text Add to dashboard Cite

Purpose To develop and demonstrate the efficacy of a novel head‐and‐neck multimodality image registration technique using deep‐learning‐based cross‐modality synthesis. Methods and Materials Twenty‐five head‐and‐neck patients received magnetic resonance (MR) and computed tomography (CT) (CTaligned) scans on the same day with the same immobilization. Fivefold cross validation was used with all of the MR‐CT pairs to train a neural network to generate synthetic CTs from MR images. Twenty‐four of 25 patients also had a separate CT without immobilization (CTnon‐aligned) and were used for testing. CTnon‐aligned's were deformed to the synthetic CT, and compared to CTnon‐aligned registered to MR. The same registrations were performed from MR to CTnon‐aligned and from synthetic CT to CTnon‐aligned. All registrations used B‐splines for modeling the deformation, and mutual information for the objective. Results were evaluated using the 95% Hausdorff distance among spinal cord contours, landmark error, inverse consistency, and Jacobian determinant of the estimated deformation fields. Results When large initial rigid misalignment is present, registering CT to MRI‐derived synthetic CT aligns the cord better than a direct registration. The average landmark error decreased from 9.8 ± 3.1 mm in MR→CTnon‐aligned to 6.0 ± 2.1 mm in CTsynth→CTnon‐aligned deformable registrations. In the CT to MR direction, the landmark error decreased from 10.0 ± 4.3 mm in CTnon‐aligned→MR deformable registrations to 6.6 ± 2.0 mm in CTnon‐aligned→CTsynth deformable registrations. The Jacobian determinant had an average value of 0.98. The proposed method also demonstrated improved inverse consistency over the direct method. Conclusions We showed that using a deep learning‐derived synthetic CT in lieu of an MR for MR→CT and CT→MR deformable registration offers superior results to direct multimodal registration.

show abstract

An Unsupervised Learning Model for Deformable Medical Image Registration

Cited by 647 publications

References 46 publications

Fast contour propagation for MR‐guided prostate radiotherapy using convolutional neural networks

Fast contour propagation for MR‐guided prostate radiotherapy using convolutional neural networks

Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning Based Registration

Multimodality image registration in the head‐and‐neck using a deep learning‐derived synthetic CT as a bridge

Contact Info

Product

Resources

About