ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration

Dey, Neel; Schlemper, Jo; Salehi, Seyed Sadegh Mohseni; Zhang, Bo; Gerig, Guido; Sofka, Michal

doi:10.1007/978-3-031-16446-0_7

Cited by 11 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also compared the performance of the ConvUNet-DIR with the current state-of-the-art supervised and unsupervised deep learning-based approaches reported in the literature utilized public brain MRI datasets. Our method (Dice = 0.980) outperformed Han et al [19] (Dice = 0.839), Martin et al [28] (Dice = 0.756), Wu et al [29] (Dice = 0.873), Meng et al [30] (Dice = 0.654), Kuang and Schmah [9] (Dice = 0.533), Mok and Chung [31] (Dice = 0.770), Huang et al [32] (Dice = 0.707), Xu et al [33] (Dice = 0.830), Dey et al [34] (Dice = 0.781), Fan et al [35] (Dice = 0.788), Liu et al [36] (Dice = 0.909), Chen et al [37] (Dice = 0.873), Wang et al [38] (Dice = 0.731) methods using unsupervised approach; and Zhu et al [39] (Dice = 0.637) method using supervised approach.…”

Section: Discussionmentioning

confidence: 99%

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

Osman,

Al-Mugren,

Tamam

et al. 2024

Radiat Oncol

View full text Add to dashboard Cite

Purpose Accurate deformable registration of magnetic resonance imaging (MRI) scans containing pathologies is challenging due to changes in tissue appearance. In this paper, we developed a novel automated three-dimensional (3D) convolutional U-Net based deformable image registration (ConvUNet-DIR) method using unsupervised learning to establish correspondence between baseline pre-operative and follow-up MRI scans of patients with brain glioma. Methods This study involved multi-parametric brain MRI scans (T1, T1-contrast enhanced, T2, FLAIR) acquired at pre-operative and follow-up time for 160 patients diagnosed with glioma, representing the BraTS-Reg 2022 challenge dataset. ConvUNet-DIR, a deep learning-based deformable registration workflow using 3D U-Net style architecture as a core, was developed to establish correspondence between the MRI scans. The workflow consists of three components: (1) the U-Net learns features from pairs of MRI scans and estimates a mapping between them, (2) the grid generator computes the sampling grid based on the derived transformation parameters, and (3) the spatial transformation layer generates a warped image by applying the sampling operation using interpolation. A similarity measure was used as a loss function for the network with a regularization parameter limiting the deformation. The model was trained via unsupervised learning using pairs of MRI scans on a training data set (n = 102) and validated on a validation data set (n = 26) to assess its generalizability. Its performance was evaluated on a test set (n = 32) by computing the Dice score and structural similarity index (SSIM) quantitative metrics. The model’s performance also was compared with the baseline state-of-the-art VoxelMorph (VM1 and VM2) learning-based algorithms. Results The ConvUNet-DIR model showed promising competency in performing accurate 3D deformable registration. It achieved a mean Dice score of 0.975 ± 0.003 and SSIM of 0.908 ± 0.011 on the test set (n = 32). Experimental results also demonstrated that ConvUNet-DIR outperformed the VoxelMorph algorithms concerning Dice (VM1: 0.969 ± 0.006 and VM2: 0.957 ± 0.008) and SSIM (VM1: 0.893 ± 0.012 and VM2: 0.857 ± 0.017) metrics. The time required to perform a registration for a pair of MRI scans is about 1 s on the CPU. Conclusions The developed deep learning-based model can perform an end-to-end deformable registration of a pair of 3D MRI scans for glioma patients without human intervention. The model could provide accurate, efficient, and robust deformable registration without needing pre-alignment and labeling. It outperformed the state-of-the-art VoxelMorph learning-based deformable registration algorithms and other supervised/unsupervised deep learning-based methods reported in the literature.

show abstract

Section: Discussionmentioning

confidence: 99%

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

Osman,

Al-Mugren,

Tamam

et al. 2024

Radiat Oncol

View full text Add to dashboard Cite

show abstract

“…Similarly, feeding two different images was used in stereo depth estimation for surgery videos [25]. Additionally, contrastive learning was applied to the unsupervised multimodal MRI registration task as a representation learning approach [26].…”

Section: Background and Related Workmentioning

confidence: 99%

3DTINC: Time-Equivariant Non-Contrastive Learning for Predicting Disease Progression from Longitudinal OCTs

Emre,

Chakravarty,

Rivail

et al. 2024

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Self-supervised learning (SSL) has emerged as a powerful technique for improving the efficiency and effectiveness of deep learning models. Contrastive methods are a prominent family of SSL that extract similar representations of two augmented views of an image while pushing away others in the representation space as negatives. However, the state-of-the-art contrastive methods require large batch sizes and augmentations designed for natural images that are impractical for 3D medical images. To address these limitations, we propose a new longitudinal SSL method, 3DTINC, based on non-contrastive learning. It is designed to learn perturbation-invariant features for 3D optical coherence tomography (OCT) volumes, using augmentations specifically designed for OCT. We introduce a new non-contrastive similarity loss term that learns temporal information implicitly from intra-patient scans acquired at different times. Our experiments show that this temporal information is crucial for predicting progression of retinal diseases, such as age-related macular degeneration (AMD). After pretraining with 3DTINC, we evaluated the learned representations and the prognostic models on two largescale longitudinal datasets of retinal OCTs where we predict the conversion to wet-AMD within a six-month interval. Our results demonstrate that each component of our contributions is crucial for learning meaningful representations useful in predicting disease progression from longitudinal volumetric scans.

show abstract

“…GLCNet [20] introduced a remote sensing semantic segmentation approach based on global style and local matching network, incorporating a matching contrastive loss to learn pixel-level information. ContraReg [21] achieved non-rigid multimodal image alignment by projecting the learned multiscale local patch features into the jointly learned inter-domain embedding domain. Li et al [22] proposed a template matching method based on contrastive learning, which increases the matching number at finer details and performs intensive learning at the pixel level.…”

Section: Introductionmentioning

confidence: 99%

Contrastive Learning of Multimodal Consistency Feature Representation for Remote Sensing Image Registration

Han,

Lv,

Wang

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Feature representation is a crucial issue in multimodal image registration. The handcrafted features extracted by traditional methods are highly sensitive to nonlinear radiation differences, while supervised learning methods are limited by deficient labeled samples in the remote sensing field. Therefore, this paper proposes a consistency feature representation learning method for multimodal image registration, which involves mapping data into a common feature space to realize the accurate alignment of remote sensing images. First, a contrastive network with a spatial attention mechanism is driven to enhance the capability to highlight high-level features of images. Second, a positive sample augmentation strategy is implemented with contrastive loss, which helps the model learn the inherent features better, and imposes constraints on the sample similarity to optimize the feature projection. Finally, a multimodal image registration framework is proposed to enhance the stability of feature matching. The proposed framework achieves accurate feature extraction and consistency feature description for multimodal images, ensuring robustness against nonlinear radiometric differences. Experimental results demonstrate that the proposed method obtains more reliable registration results on the SEN1-2 dataset. Furthermore, the proposed algorithm achieves superior performance on data from other modalities, indicating strong generalization ability.

show abstract

ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration

Cited by 11 publications

References 32 publications

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

3DTINC: Time-Equivariant Non-Contrastive Learning for Predicting Disease Progression from Longitudinal OCTs

Contrastive Learning of Multimodal Consistency Feature Representation for Remote Sensing Image Registration

Contact Info

Product

Resources

About