Joint affine and deformable three‐dimensional networks for brain MRI registration

Abstract

Purpose Volumetric medical image registration has important clinical significance. Traditional registration methods may be time‐consuming when processing large volumetric data due to their iterative optimizations. In contrast, existing deep learning‐based networks can obtain the registration quickly. However, most of them require independent rigid alignment before deformable registration; these two steps are often performed separately and cannot be end‐to‐end. Methods We propose an end‐to‐end joint affine and deformable network for three‐dimensional (3D) medical image registration. The proposed network combines two deformation methods; the first one is for obtaining affine alignment and the second one is a deformable subnetwork for achieving the nonrigid registration. The parameters of the two subnetworks are shared. The global and local similarity measures are used as loss functions for the two subnetworks, respectively. Moreover, an anatomical similarity loss is devised to weakly supervise the training of the whole registration network. Finally, the trained network can perform deformable registration in one forward pass. Results The efficacy of our network was extensively evaluated on three public brain MRI datasets including Mindboggle101, LPBA40, and IXI. Experimental results demonstrate our network consistently outperformed several state‐of‐the‐art methods with respect to the metrics of Dice index (DSC), Hausdorff distance (HD), and average symmetric surface distance (ASSD). Conclusions The proposed network provides accurate and robust volumetric registration without any pre‐alignment requirement, which facilitates the end‐to‐end deformable registration.

Salient deformable network for abdominal multiorgan registration

“…We compared our method with four cutting‐edge registration networks, including VoxelMorph (VM), 15 FAIM, 22 Multi‐FC, 23 and JADN 24 . Among these compared methods, VM, FAIM, and Multi‐FC are unsupervised methods using only intensity‐based similarity to train their registration networks.…”

“…The deformable subnetwork has an encoder‐decoder architecture, and includes a dilated convolutional block at the last layer of the encoder. The loss function for the registration backbone is defined as $${\pounds}_{reg}$$: $$\begin{equation} \begin{split} {\pounds}_{reg}(F, M) = & - {\pounds}_{affine}(F, M\circ \phi _{affine})\\ &- {\pounds}_{deformable}(F, M \circ \phi _{affine}\circ \phi _{deformable})\\ & + \lambda {\pounds}_{smooth}(\phi _{deformable}), \end{split} \end{equation}$$where $${\pounds}_{affine}$$ and $${\pounds}_{deformable}$$ define similarity criterion (here, we use normalized cross‐correlation function 3 ), and $${\pounds}_{smooth}$$ regularizes the smoothness of the deformable displacement field $$\phi _{deformable}$$ (here, we use a diffusion regularizer 24 ); operator ○ is a warping operation; λ is a weighting parameter.…”

“…The general idea is to assist the registration process by introducing anatomical information, for example, the extracted corresponding anatomical landmarks or structures. Zhu et al 24 developed a weakly supervised registration network via an additional shape similarity loss to improve registration performance over unsupervised training. Hu et al 25 proposed a weakly supervised, label‐driven formulation to learn 3D voxel correspondence from higher‐level label correspondence, thereby bypassing classical intensity‐based image similarity measures.…”

A review of deep learning-based three-dimensional medical image registration methods

“…All the above-described networks achieved promising registration accuracy across various datasets. Moreover, aside from purely deformable registration, Zhu et al used a registration scheme that combined both affine and deformable MR-MR brain registration methods (101). The affine network used global similarity as the loss function, whereas the deformable network used local similarity.…”

Variance‐aware attention U‐Net for multi‐organ segmentation

“…Deep neural networks have achieved outstanding achievements in many areas of medical image computing, 1 including de‐noising, 9,10 super‐resolution, 11,12 cancer detection, 13 registration, 14 and single organ segmentation (e.g., heart, 15 prostate, 16 liver 17 ). However, multi‐organ segmentation is still a challenging task (e.g., abdominal multi‐organs 18 ).…”