A deep learning based framework for the registration of three dimensional multi-modal medical images of the head

Islam, Kh Tohidul; Wijewickrema, Sudanthi; O’Leary, Stephen

doi:10.1038/s41598-021-81044-7

Cited by 30 publications

(13 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used a modified U-Net [30,31] architecture, as it has been shown to perform well in similar tasks [27,[32][33][34][35]. Prior to training, we co-registered each image pair using an existing multi-modal image registration method [36]. To train the 3-D CNN, we used the volumes themselves, while we used axial slices of the volumes to train the 2-D CNN.…”

Section: Synthetic Ct Generationmentioning

confidence: 99%

A Deep Learning Framework for Segmenting Brain Tumors Using MRI and Synthetically Generated CT Images

Islam

Wijewickrema

O’Leary

2022

Sensors

Self Cite

View full text Add to dashboard Cite

Multi-modal three-dimensional (3-D) image segmentation is used in many medical applications, such as disease diagnosis, treatment planning, and image-guided surgery. Although multi-modal images provide information that no single image modality alone can provide, integrating such information to be used in segmentation is a challenging task. Numerous methods have been introduced to solve the problem of multi-modal medical image segmentation in recent years. In this paper, we propose a solution for the task of brain tumor segmentation. To this end, we first introduce a method of enhancing an existing magnetic resonance imaging (MRI) dataset by generating synthetic computed tomography (CT) images. Then, we discuss a process of systematic optimization of a convolutional neural network (CNN) architecture that uses this enhanced dataset, in order to customize it for our task. Using publicly available datasets, we show that the proposed method outperforms similar existing methods.

show abstract

Section: Synthetic Ct Generationmentioning

confidence: 99%

A Deep Learning Framework for Segmenting Brain Tumors Using MRI and Synthetically Generated CT Images

Islam

Wijewickrema

O’Leary

2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…DL‐generated synthetic CT (sCT) can be used as MRI's surrogate to convert the MRI‐CT DIR into mono‐modality image registration for head and neck radiotherapy 29 . Islam et al 30 . introduced a fully automated DL framework for 3D multi‐modal medical image registration for CT‐MRI of the head.…”

Section: Image Selection and Registrationmentioning

confidence: 99%

“…DL-generated synthetic CT (sCT) can be used as MRI's surrogate to convert the MRI-CT DIR into mono-modality image registration for head and neck radiotherapy. 29 Islam et al 30 introduced a fully automated DL framework for 3D multi-modal medical image registration for CT-MRI of the head. The application of DL methods for image registration is still at the research phase but has great potential for clinical usage.…”

Section: Mpmri Image Registrationmentioning

confidence: 99%

Multi‐parametric magnetic resonance imaging for radiation treatment planning

et al. 2022

View full text Add to dashboard Cite

In recent years, multi-parametric magnetic resonance imaging (MpMRI) has played a major role in radiation therapy treatment planning. The superior soft tissue contrast, functional or physiological imaging capabilities, and the flexibility of site-specific image sequence development has placed MpMRI at the forefront. In this article, the present status of MpMRI for external beam radiation therapy planning is reviewed. Common MpMRI sequences, preprocessing, and quality assurance strategies are briefly discussed, and various image registration techniques and strategies are addressed. Image segmentation methods including automatic segmentation and deep learning techniques for organs at risk and target delineation are reviewed. Due to the advancement in MRI-guided online adaptive radiotherapy, treatment planning considerations addressing MRI only planning are also discussed. K E Y W O R D S MpMRI, MRI for treatment planning, MRI-guided radiotherapy 1 2 IMAGE SELECTION AND REGISTRATION Brief description of common MpMRI sequencesMpMRI sequences typically include multi-slice 2D and 3D images with T1 and T2 weighting and a variety of 2836

show abstract

“…Methods based on direct prediction of the transformation have so far mostly provided an improvement of the run-time, while exhibiting registration performance that is (at best) on par with traditional methods [ 10 , 11 ]. A related, but different recent method for multimodal rigid registration [ 12 ] uses a large neural network as a feature extractor, training a network to recover a transformation from a single image. This method, however, fails in case a canonical space does not exist, which holds for a majority of the imaging scenarios considered in this study.…”

Section: Introductionmentioning

confidence: 99%

Is image-to-image translation the panacea for multimodal image registration? A comparative study

Öfverstedt

Lindblad

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

Despite current advancement in the field of biomedical image processing, propelled by the deep learning revolution, multimodal image registration, due to its several challenges, is still often performed manually by specialists. The recent success of image-to-image (I2I) translation in computer vision applications and its growing use in biomedical areas provide a tempting possibility of transforming the multimodal registration problem into a, potentially easier, monomodal one. We conduct an empirical study of the applicability of modern I2I translation methods for the task of rigid registration of multimodal biomedical and medical 2D and 3D images. We compare the performance of four Generative Adversarial Network (GAN)-based I2I translation methods and one contrastive representation learning method, subsequently combined with two representative monomodal registration methods, to judge the effectiveness of modality translation for multimodal image registration. We evaluate these method combinations on four publicly available multimodal (2D and 3D) datasets and compare with the performance of registration achieved by several well-known approaches acting directly on multimodal image data. Our results suggest that, although I2I translation may be helpful when the modalities to register are clearly correlated, registration of modalities which express distinctly different properties of the sample are not well handled by the I2I translation approach. The evaluated representation learning method, which aims to find abstract image-like representations of the information shared between the modalities, manages better, and so does the Mutual Information maximisation approach, acting directly on the original multimodal images. We share our complete experimental setup as open-source (https://github.com/MIDA-group/MultiRegEval), including method implementations, evaluation code, and all datasets, for further reproducing and benchmarking.

show abstract

A deep learning based framework for the registration of three dimensional multi-modal medical images of the head

Cited by 30 publications

References 66 publications

A Deep Learning Framework for Segmenting Brain Tumors Using MRI and Synthetically Generated CT Images

A Deep Learning Framework for Segmenting Brain Tumors Using MRI and Synthetically Generated CT Images

Multi‐parametric magnetic resonance imaging for radiation treatment planning

Is image-to-image translation the panacea for multimodal image registration? A comparative study

Contact Info

Product

Resources

About