Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

Elmahdy, Mohamed S.; Wolterink, Jelmer M.; Sokooti, Hessam; Išgum, Ivana; Staring, Marius

doi:10.1007/978-3-030-32226-7_41

Cited by 21 publications

(21 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…al. [7], a generative network is trained for contour propagation by registration, while a discrimination network evaluates the quality of the propagated contours. Finally, we compare our methods against the hybrid method proposed by Elmahdy et.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…In the paper, we compared our algorithm against different algorithms from various categories: non-learning (elastix [49], a popular conventional tool); hybrid [6], and GAN-based [7]. The presented multi-task networks outperformed these approaches on the validation set and performed on par to these methods for the test set.…”

Section: Discussionmentioning

confidence: 99%

“…The authors integrated domain-specific knowledge such as air pocket inpainting as well as contrast clipping, moreover they added an extra registration step in order to focus on the seminal vesicles and rectum. Elmahdy et al [7] and Mahapatra et al [33] proposed a GAN-based (Generative Adversarial Network) approach, where a generative network predicts the correspondence between a pair of images and a discriminator network for giving feedback on the quality of the deformed contours. Exemplar methods of the second category are Xu et al [34], who presented a framework that simultaneously trains a registration and a segmentation network.…”

Section: A Related Workmentioning

confidence: 99%

“…Image registration is the task of finding the geometrical correspondence between images that were acquired at different time steps or from different imaging modalities. These two tasks are complementary, as for example image atlases warped by image registration algorithms are often used for image segmentation [4], [5], while image contours can be used to guide the image registration method in addition to the intensity images [6], [7], [8]. Contours are also used for evaluating the quality of the registration [9], [10].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer

et al. 2021

Self Cite

View full text Add to dashboard Cite

Medical image registration and segmentation are two of the most frequent tasks in medical image analysis. As these tasks are complementary and correlated, it would be beneficial to apply them simultaneously in a joint manner. In this paper, we formulate registration and segmentation as a joint problem via a Multi-Task Learning (MTL) setting, allowing these tasks to leverage their strengths and mitigate their weaknesses through the sharing of beneficial information. We propose to merge these tasks not only on the loss level, but on the architectural level as well. We studied this approach in the context of adaptive image-guided radiotherapy for prostate cancer, where planning and follow-up CT images as well as their corresponding contours are available for training. At testing time the contours of the follow-up scans are not available, which is a common scenario in adaptive radiotherapy. The study involves two datasets from different manufacturers and institutes. The first dataset was divided into training (12 patients) and validation (6 patients), and was used to optimize and validate the methodology, while the second dataset (14 patients) was used as an independent test set. We carried out an extensive quantitative comparison between the quality of the automatically generated contours from different network architectures as well as loss weighting methods. Moreover, we evaluated the quality of the generated deformation vector field (DVF). We show that MTL algorithms outperform their Single-Task Learning (STL) counterparts and achieve better generalization on the independent test set. The best algorithm achieved a mean surface distance of 1.06 ± 0.3 mm, 1.27 ± 0.4 mm, 0.91 ± 0.4 mm, and 1.76 ± 0.8 mm on the validation set for the prostate, seminal vesicles, bladder, and rectum, respectively. The high accuracy of the proposed method combined with the fast inference speed, makes it a promising method for automatic re-contouring of follow-up scans for adaptive radiotherapy, potentially reducing treatment related complications and therefore improving patients quality-of-life after treatment.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…But this study only used CycleGAN for image synthesis, not for image registration directly. Elmahdy et al 24 used unsupervised GANs for joint registration in prostate CT radiotherapy; however, their method was not suitable for multi‐modal image registration because they synthesized real samples through artificial deformations which are not useful for multi‐modal image registration. Kim et al 25 proposed a cycle‐consistent CNN to register multiphase liver CT images, but their method was also not suitable for CT‐MR registration because the loss functions they used could not evaluate the similarity between CT and MR images.…”

Section: Introductionmentioning

confidence: 99%

End‐to‐end unsupervised cycle‐consistent fully convolutional network for 3D pelvic CT‐MR deformable registration

Guo

Wang

et al. 2020

J Applied Clin Med Phys

View full text Add to dashboard Cite

Objective To improve the efficiency of computed tomography (CT)‐magnetic resonance (MR) deformable image registration while ensuring the registration accuracy. Methods Two fully convolutional networks (FCNs) for generating spatial deformable grids were proposed using the Cycle‐Consistent method to ensure the deformed image consistency with the reference image data. In all, 74 pelvic cases consisting of both MR and CT images were studied, among which 64 cases were used as training data and 10 cases as the testing data. All training data were standardized and normalized, following simple image preparation to remove the redundant air. Dice coefficients and average surface distance (ASD) were calculated for regions of interest (ROI) of CT‐MR image pairs, before and after the registration. The performance of the proposed method (FCN with Cycle‐Consistent) was compared with that of Elastix software, MIM software, and FCN without cycle‐consistent. Results The results show that the proposed method achieved the best performance among the four registration methods tested in terms of registration accuracy and the method was more stable than others in general. In terms of average registration time, Elastix took 64 s, MIM software took 28 s, and the proposed method was found to be significantly faster, taking <0.1 s. Conclusion The proposed method not only ensures the accuracy of deformable image registration but also greatly reduces the time required for image registration and improves the efficiency of the registration process. In addition, compared with other deep learning methods, the proposed method is completely unsupervised and end‐to‐end.

show abstract

Salient deformable network for abdominal multiorgan registration

2022

View full text Add to dashboard Cite

Background: Image registration has long been an active research area in the society of medical image computing, which is to perform spatial transformation between a pair of images and establish a point-wise correspondence to achieve spatial consistency. Purpose: Previous work mainly focused on learning complicated deformation fields by maximizing the global-level (i.e., foreground plus background) image similarity.We argue that taking the background similarity into account may not be a good solution, if we only seek the accurate alignment of target organs/regions in real clinical practice. Methods: We, therefore, propose a novel concept of Salient Registration and introduce a novel deformable network equipped with a saliency module. Specifically, a multitask learning-based saliency module is proposed to discriminate the salient regions-of -registration in a semisupervised manner. Then, our deformable network analyzes the intensity and anatomical similarity of salient regions, and finally conducts the salient deformable registration. Results: We evaluate the efficacy of the proposed network on challenging abdominal multiorgan CT scans. The experimental results demonstrate that the proposed registration network outperforms other state-of -the-art methods, achieving a mean Dice similarity coefficient (DSC) of 40.2%,Hausdorff distance (95 HD) of 20.8 mm, and average symmetric surface distance (ASSD) of 4.58 mm. Moreover, even by training using one labeled data, our network can still attain satisfactory registration performance, with a mean DSC of 39.2%, 95 HD of 21.2 mm, and ASSD of 4.78 mm. Conclusions:The proposed network provides an accurate solution for multiorgan registration and has the potential to be used for improving other registration applications. The code is publicly available at https://github.com/Rrrfrr/Salient-Deformable-Network.

show abstract

Adversarial Optimization for Joint Registration and Segmentation in Prostate CT Radiotherapy

Cited by 21 publications

References 16 publications

Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer

Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer

End‐to‐end unsupervised cycle‐consistent fully convolutional network for 3D pelvic CT‐MR deformable registration

Salient deformable network for abdominal multiorgan registration

Contact Info

Product

Resources

About