One-Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking

Fechter, Tobias; Baltas, Dimos

doi:10.1109/tmi.2020.2972616

Cited by 87 publications

(53 citation statements)

References 37 publications

(40 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different from many supervised deep learning methods, we trained the LungRegNet in a completely unsupervised 27 De Vos et al 25 Sentker et al 26 Fechter et al 34 Sokooti et al 33 Jiang et al 36 LungRegNet-v3 manner. One benefit of unsupervised training is that it could mitigate the problem of unavailability of ground truth.…”

Section: Discussionmentioning

confidence: 99%

“…Fechter et al proposed a one-shot learning for DIR and periodic motion tracking. 34 They employed a UNet with multiresolution approach for coarseto-fine image registration. The network was trained in an unsupervised manner, meaning no known transformation was required.…”

Section: Related Workmentioning

confidence: 99%

“…Fechter et al . proposed a one‐shot learning for DIR and periodic motion tracking . They employed a UNet with multiresolution approach for coarse‐to‐fine image registration.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

LungRegNet: An unsupervised deformable image registration method for 4D‐CT lung

Lei

Wang

et al. 2020

Medical Physics

View full text Add to dashboard Cite

Purpose: To develop an accurate and fast deformable image registration (DIR) method for fourdimensional computed tomography (4D-CT) lung images. Deep learning-based methods have the potential to quickly predict the deformation vector field (DVF) in a few forward predictions. We have developed an unsupervised deep learning method for 4D-CT lung DIR with excellent performances in terms of registration accuracies, robustness, and computational speed. Methods: A fast and accurate 4D-CT lung DIR method, namely LungRegNet, was proposed using deep learning. LungRegNet consists of two subnetworks which are CoarseNet and FineNet. As the name suggests, CoarseNet predicts large lung motion on a coarse scale image while FineNet predicts local lung motion on a fine scale image. Both the CoarseNet and FineNet include a generator and a discriminator. The generator was trained to directly predict the DVF to deform the moving image. The discriminator was trained to distinguish the deformed images from the original images. Coarse-Net was first trained to deform the moving images. The deformed images were then used by the Fine-Net for FineNet training. To increase the registration accuracy of the LungRegNet, we generated vessel-enhanced images by generating pulmonary vasculature probability maps prior to the network prediction. Results: We performed fivefold cross validation on ten 4D-CT datasets from our department. To compare with other methods, we also tested our method using separate 10 DIRLAB datasets that provide 300 manual landmark pairs per case for target registration error (TRE) calculation. Our results suggested that LungRegNet has achieved better registration accuracy in terms of TRE than other deep learning-based methods available in the literature on DIRLAB datasets. Compared to conventional DIR methods, LungRegNet could generate comparable registration accuracy with TRE smaller than 2 mm. The integration of both the discriminator and pulmonary vessel enhancements into the network was crucial to obtain high registration accuracy for 4D-CT lung DIR. The mean and standard deviation of TRE were 1.00 AE 0.53 mm and 1.59 AE 1.58 mm on our datasets and DIRLAB datasets respectively. Conclusions: An unsupervised deep learning-based method has been developed to rapidly and accurately register 4D-CT lung images. LungRegNet has outperformed its deep-learning-based peers and achieved excellent registration accuracy in terms of TRE.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

LungRegNet: An unsupervised deformable image registration method for 4D‐CT lung

Lei

Wang

et al. 2020

Medical Physics

View full text Add to dashboard Cite

show abstract

“…Hence, several studies have focused on designing invertible frameworks and appropriate losses to tackle this issue. Using a cyclic constraint in the loss, Fechter et al [59] presented an approach to calculate DVF for periodic motion tracking in 3D and 4D medical image datasets. This approach was able to calculate the forward and inverse transformation simultaneously.…”

Section: Invertibility Regularisermentioning

confidence: 99%

“…Besides, there are also several multi-modal datasets containing CT images, VISCERAL Anatomy3 [114], MM-WHS [115] and RIRE respectively. We found that CT image registration is the second largest domain used for developing medical image registration methods, with numerous recent studies on the topic [31,32,41,55,59,69]. Compared with brain MRI registration, CT image registration is more challenging to some extent, due to limited soft-tissue contrast, and greater variability in image quality.…”

Section: Ct Registrationmentioning

confidence: 99%

Deep learning in medical image registration

et al. 2020

View full text Add to dashboard Cite

Image registration is a fundamental task in multiple medical image analysis applications. With the advent of deep learning, there have been significant advances in algorithmic performance for various computer vision tasks in recent years, including medical image registration. The last couple of years have seen a dramatic increase in the development of deep learning-based medical image registration algorithms. Consequently, a comprehensive review of the current state-of-the-art algorithms in the field is timely, and necessary. This review is aimed at understanding the clinical applications and challenges that drove this innovation, analysing the functionality and limitations of existing approaches, and at providing insights to open challenges and as yet unmet clinical needs that could shape future research directions. To this end, the main contributions of this paper are: (a) discussion of all deep learning-based medical image registration papers published since 2013 with significant methodological and/or functional contributions to the field; (b) analysis of the development and evolution of deep learning-based image registration methods, summarising the current trends and challenges in the domain; and (c) overview of unmet clinical needs and potential directions for future research in deep learning-based medical image registration.

show abstract

Unsupervised deep learning registration model for multimodal brain images

Abbasi,

Mehdizadeh,

Boveiri

et al. 2023

J Applied Clin Med Phys

View full text Add to dashboard Cite

Multimodal image registration is a key for many clinical image‐guided interventions. However, it is a challenging task because of complicated and unknown relationships between different modalities. Currently, deep supervised learning is the state‐of‐theart method at which the registration is conducted in end‐to‐end manner and one‐shot. Therefore, a huge ground‐truth data is required to improve the results of deep neural networks for registration. Moreover, supervised methods may yield models that bias towards annotated structures. Here, to deal with above challenges, an alternative approach is using unsupervised learning models. In this study, we have designed a novel deep unsupervised Convolutional Neural Network (CNN)‐based model based on computer tomography/magnetic resonance (CT/MR) co‐registration of brain images in an affine manner. For this purpose, we created a dataset consisting of 1100 pairs of CT/MR slices from the brain of 110 neuropsychic patients with/without tumor. At the next step, 12 landmarks were selected by a well‐experienced radiologist and annotated on each slice resulting in the computation of series of metrics evaluation, target registration error (TRE), Dice similarity, Hausdorff, and Jaccard coefficients. The proposed method could register the multimodal images with TRE 9.89, Dice similarity 0.79, Hausdorff 7.15, and Jaccard 0.75 that are appreciable for clinical applications. Moreover, the approach registered the images in an acceptable time 203 ms and can be appreciable for clinical usage due to the short registration time and high accuracy. Here, the results illustrated that our proposed method achieved competitive performance against other related approaches from both reasonable computation time and the metrics evaluation.

show abstract

One-Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking

Cited by 87 publications

References 37 publications

LungRegNet: An unsupervised deformable image registration method for 4D‐CT lung

LungRegNet: An unsupervised deformable image registration method for 4D‐CT lung

Deep learning in medical image registration

Unsupervised deep learning registration model for multimodal brain images

Contact Info

Product

Resources

About