iRegNet: Non-Rigid Registration of MRI to Interventional US for Brain-Shift Compensation Using Convolutional Neural Networks

Zeineldin, Ramy A.; Karar, Mohamed Esmail; Elshaer, Ziad; Schmidhammer, Markus; Coburger, Jan; Wirtz, Christian Rainer; Burgert, Oliver; Mathis-Ullrich, Franziska

doi:10.1109/access.2021.3120306

Cited by 6 publications

(3 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The underlying idea behind deep similarity-based methods centers on using deeplearning-based similarity measures to replace the traditional similarity metrics such as the sum of square distance (SSD), mean squared distance (MSD), (normalized) cross-correlation (CC), (normalized) mutual information (MI), etc. This method has shown its feasibility in multimodal registration between CT and MR [50][51][52]; brain T1 and T2 MRI images [53][54][55]; and MRI and ultrasound [55][56][57][58].…”

Section: Image Registrationmentioning

confidence: 99%

Advances in TEE-Centric Intraprocedural Multimodal Image Guidance for Congenital and Structural Heart Disease

Zhang,

Gosnell,

Nainamalai

et al. 2023

Diagnostics

View full text Add to dashboard Cite

Percutaneous interventions are gaining rapid acceptance in cardiology and revolutionizing the treatment of structural heart disease (SHD). As new percutaneous procedures of SHD are being developed, their associated complexity and anatomical variability demand a high-resolution special understanding for intraprocedural image guidance. During the last decade, three-dimensional (3D) transesophageal echocardiography (TEE) has become one of the most accessed imaging methods for structural interventions. Although 3D-TEE can assess cardiac structures and functions in real-time, its limitations (e.g., limited field of view, image quality at a large depth, etc.) must be addressed for its universal adaptation, as well as to improve the quality of its imaging and interventions. This review aims to present the role of TEE in the intraprocedural guidance of percutaneous structural interventions. We also focus on the current and future developments required in a multimodal image integration process when using TEE to enhance the management of congenital and SHD treatments.

show abstract

Section: Image Registrationmentioning

confidence: 99%

Advances in TEE-Centric Intraprocedural Multimodal Image Guidance for Congenital and Structural Heart Disease

Zhang,

Gosnell,

Nainamalai

et al. 2023

Diagnostics

View full text Add to dashboard Cite

show abstract

“…Recent developments in deep learning, specifically convolutional neural networks (CNN) have achieved excellent performance for processing and analyzing medical images, including those associated with brain tumor segmentation 8 , 9 , image registration 10 , 11 , and image classification 12 . In particular, the convolutional encoder-decoder architectures, U-Net 13 , 14 , have revolutionized the medical field with outstanding feature representation capabilities.…”

Section: Introductionmentioning

confidence: 99%

Explainable hybrid vision transformers and convolutional network for multimodal glioma segmentation in brain MRI

Zeineldin,

Karar,

Elshaer

et al. 2024

Sci Rep

Self Cite

View full text Add to dashboard Cite

Accurate localization of gliomas, the most common malignant primary brain cancer, and its different sub-region from multimodal magnetic resonance imaging (MRI) volumes are highly important for interventional procedures. Recently, deep learning models have been applied widely to assist automatic lesion segmentation tasks for neurosurgical interventions. However, these models are often complex and represented as “black box” models which limit their applicability in clinical practice. This article introduces new hybrid vision Transformers and convolutional neural networks for accurate and robust glioma segmentation in Brain MRI scans. Our proposed method, TransXAI, provides surgeon-understandable heatmaps to make the neural networks transparent. TransXAI employs a post-hoc explanation technique that provides visual interpretation after the brain tumor localization is made without any network architecture modifications or accuracy tradeoffs. Our experimental findings showed that TransXAI achieves competitive performance in extracting both local and global contexts in addition to generating explainable saliency maps to help understand the prediction of the deep network. Further, visualization maps are obtained to realize the flow of information in the internal layers of the encoder-decoder network and understand the contribution of MRI modalities in the final prediction. The explainability process could provide medical professionals with additional information about the tumor segmentation results and therefore aid in understanding how the deep learning model is capable of processing MRI data successfully. Thus, it enables the physicians’ trust in such deep learning systems towards applying them clinically. To facilitate TransXAI model development and results reproducibility, we will share the source code and the pre-trained models after acceptance at https://github.com/razeineldin/TransXAI.

show abstract

“…Few studies have addressed this correspondence utilizing different convolutional neural networks (CNNs) for deformable registration. These methods include proposing a one-stage [15][16][17][18][19], two-stage [20] or three-stage [21] registration pipeline for this purpose. Despite the inspiring performance of these models, obtaining accurate results may remain challenging for deep learning-based deformable registration because of the large deformation of the healthy images.…”

Section: Introductionmentioning

confidence: 99%

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

Osman,

Al-Mugren,

Tamam

et al. 2024

Radiat Oncol

View full text Add to dashboard Cite

Purpose Accurate deformable registration of magnetic resonance imaging (MRI) scans containing pathologies is challenging due to changes in tissue appearance. In this paper, we developed a novel automated three-dimensional (3D) convolutional U-Net based deformable image registration (ConvUNet-DIR) method using unsupervised learning to establish correspondence between baseline pre-operative and follow-up MRI scans of patients with brain glioma. Methods This study involved multi-parametric brain MRI scans (T1, T1-contrast enhanced, T2, FLAIR) acquired at pre-operative and follow-up time for 160 patients diagnosed with glioma, representing the BraTS-Reg 2022 challenge dataset. ConvUNet-DIR, a deep learning-based deformable registration workflow using 3D U-Net style architecture as a core, was developed to establish correspondence between the MRI scans. The workflow consists of three components: (1) the U-Net learns features from pairs of MRI scans and estimates a mapping between them, (2) the grid generator computes the sampling grid based on the derived transformation parameters, and (3) the spatial transformation layer generates a warped image by applying the sampling operation using interpolation. A similarity measure was used as a loss function for the network with a regularization parameter limiting the deformation. The model was trained via unsupervised learning using pairs of MRI scans on a training data set (n = 102) and validated on a validation data set (n = 26) to assess its generalizability. Its performance was evaluated on a test set (n = 32) by computing the Dice score and structural similarity index (SSIM) quantitative metrics. The model’s performance also was compared with the baseline state-of-the-art VoxelMorph (VM1 and VM2) learning-based algorithms. Results The ConvUNet-DIR model showed promising competency in performing accurate 3D deformable registration. It achieved a mean Dice score of 0.975 ± 0.003 and SSIM of 0.908 ± 0.011 on the test set (n = 32). Experimental results also demonstrated that ConvUNet-DIR outperformed the VoxelMorph algorithms concerning Dice (VM1: 0.969 ± 0.006 and VM2: 0.957 ± 0.008) and SSIM (VM1: 0.893 ± 0.012 and VM2: 0.857 ± 0.017) metrics. The time required to perform a registration for a pair of MRI scans is about 1 s on the CPU. Conclusions The developed deep learning-based model can perform an end-to-end deformable registration of a pair of 3D MRI scans for glioma patients without human intervention. The model could provide accurate, efficient, and robust deformable registration without needing pre-alignment and labeling. It outperformed the state-of-the-art VoxelMorph learning-based deformable registration algorithms and other supervised/unsupervised deep learning-based methods reported in the literature.

show abstract

iRegNet: Non-Rigid Registration of MRI to Interventional US for Brain-Shift Compensation Using Convolutional Neural Networks

Cited by 6 publications

References 59 publications

Advances in TEE-Centric Intraprocedural Multimodal Image Guidance for Congenital and Structural Heart Disease

Advances in TEE-Centric Intraprocedural Multimodal Image Guidance for Congenital and Structural Heart Disease

Explainable hybrid vision transformers and convolutional network for multimodal glioma segmentation in brain MRI

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

Contact Info

Product

Resources

About